-- Logs begin at Sun 2022-09-18 00:10:21 UTC, end at Sun 2022-09-18 01:17:27 UTC. -- Sep 18 00:48:04.498843 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 18 00:48:04.509329 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 18 00:48:04 2022] *** Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: nodename: dv-cinder-master-3-iscsi Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: machine: x86_64 Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: clock source: unix Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: pcre jit disabled Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: detected number of CPU cores: 4 Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: current working directory: / Sep 18 00:48:04.511832 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: detected binary path: /usr/bin/uwsgi-core Sep 18 00:48:04.512542 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: your processes number limit is 47680 Sep 18 00:48:04.512542 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: your memory page size is 4096 bytes Sep 18 00:48:04.512542 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: detected max file descriptor number: 2048 Sep 18 00:48:04.512542 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: lock engine: pthread robust mutexes Sep 18 00:48:04.512542 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: thunder lock: enabled Sep 18 00:48:04.512542 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 18 00:48:04.512542 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 18 00:48:04.551457 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: Python main interpreter initialized at 0x557d59a6b9c0 Sep 18 00:48:04.551457 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: python threads support enabled Sep 18 00:48:04.551457 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: your server socket listen backlog is limited to 100 connections Sep 18 00:48:04.551457 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: your mercy for graceful operations on workers is 90 seconds Sep 18 00:48:04.551882 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: mapped 403077 bytes (393 KB) for 2 cores Sep 18 00:48:04.551882 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: *** Operational MODE: preforking *** Sep 18 00:48:04.552163 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: *** uWSGI is running in multiple interpreter mode *** Sep 18 00:48:04.552163 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: spawned uWSGI master process (pid: 105590) Sep 18 00:48:04.552375 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 18 00:48:04.553563 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: spawned uWSGI worker 1 (pid: 105591, cores: 1) Sep 18 00:48:04.553563 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: spawned uWSGI worker 2 (pid: 105592, cores: 1) Sep 18 00:48:04.553563 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 18 00:48:05.242169 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 00:48:05.242928 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 00:48:06.333797 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:48:06.333797 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: from cryptography.utils import int_from_bytes Sep 18 00:48:06.334518 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:48:06.334518 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: from cryptography.utils import int_from_bytes Sep 18 00:48:06.334636 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:48:06.334636 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: from cryptography.utils import int_from_bytes Sep 18 00:48:06.335033 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:48:06.335033 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: from cryptography.utils import int_from_bytes Sep 18 00:48:08.817199 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 00:48:08.817773 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 00:48:08.817980 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 00:48:08.818340 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 00:48:08.818623 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 00:48:08.819010 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.819316 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 00:48:08.819712 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.819812 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 00:48:08.819903 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.819903 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.820087 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 00:48:08.820263 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.820442 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 00:48:08.820549 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.820692 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 00:48:08.820843 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.821706 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.821807 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.822030 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.822133 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.822283 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.822386 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.822622 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.822734 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.822938 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.823024 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.823254 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.823348 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.823553 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.823644 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.823857 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.823957 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.824168 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.824259 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.824422 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.824515 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.824732 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.824822 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.825048 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.825137 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.825369 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.825542 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.825938 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.825938 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.826175 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.826288 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.826400 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.826539 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.826657 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.826859 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.827038 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.827197 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.827290 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.827473 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.827693 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.827859 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.827956 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.828081 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.828239 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.828395 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.828541 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.828695 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.828872 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.829087 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.829172 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.829399 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.829488 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.829705 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.829795 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.830011 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.830137 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.830350 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.830445 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.830677 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.830805 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.831090 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.831189 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.831386 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.831485 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.831692 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.831794 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.831999 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.832090 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.832316 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.832400 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.832629 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.832723 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.832940 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.833031 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.833249 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.833340 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.833544 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.833636 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.833845 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.833935 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.834296 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.834395 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.834513 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.834589 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.834770 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.834861 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.835069 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.835170 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.835313 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.835397 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.835612 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.835703 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.835926 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.836020 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.836240 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.836330 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.836539 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.836675 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.836829 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.836990 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.837132 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.837281 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.837421 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.837576 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.837732 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.837870 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.838032 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.838185 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.838405 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.838602 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.838602 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.838897 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.839041 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.839198 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.839341 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.839519 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.839665 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.839808 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.839941 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.840085 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.840297 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.840463 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.840613 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.840774 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.840931 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.841086 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.841228 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.841386 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.841537 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.841718 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.841866 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842014 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842176 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842299 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842388 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842485 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842566 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842643 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842733 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842813 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.842889 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.843001 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.843170 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.843310 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.843615 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.843711 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.844052 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.844145 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.844400 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.844611 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.844710 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.844990 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.845302 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.845395 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.845628 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.845718 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.845927 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.846018 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.846172 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.846294 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.846519 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.846623 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.846786 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.846933 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.847097 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.847242 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.847398 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.847546 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.847698 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.847892 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.848156 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.848459 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.848549 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.848759 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.848933 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.849059 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.849188 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.849358 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.849491 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.849658 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.849793 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.849957 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.850174 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.850310 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.850406 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.850579 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.850736 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.850882 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.851141 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.851358 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.851661 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.851746 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.851977 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.852228 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.852349 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.852600 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.852823 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.852915 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.853189 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.853395 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.853481 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.853787 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.853955 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.854270 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.854360 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.854599 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.854678 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.854917 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.855054 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.855216 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.855300 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.855516 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.855609 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.855820 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.855919 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.856118 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.856207 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.856419 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.856511 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.856730 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.856820 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.857023 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.857115 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.857323 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.857412 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.857627 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.857721 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.857925 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.858007 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.858361 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.858471 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.858730 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.858959 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.859055 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.859320 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.859555 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.859646 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.859905 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.860082 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.860202 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.860495 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.860583 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.860807 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.861047 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.861142 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.861359 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.861611 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.861702 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.861915 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.862009 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.862218 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.862357 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.862613 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.862716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.862875 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.862968 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.863178 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.863271 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.863475 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.863574 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.863772 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.863858 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.864075 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.864166 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.864377 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.864461 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.864678 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.864764 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.864978 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.865064 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.865278 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.865365 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.865573 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.865665 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.865886 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.865978 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.866240 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.866347 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.866533 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.866633 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.866839 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.866925 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.867141 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.867235 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.867456 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:48:08.867540 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.867772 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.867856 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.868606 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.868761 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.868921 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.869022 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.869235 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.869326 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.869549 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.869643 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.869872 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.869978 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.870190 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.870283 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.870515 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.870616 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.870832 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.870923 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.871161 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.871247 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.871474 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.871569 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.871786 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.871871 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.872096 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.872185 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.872405 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.872526 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.872714 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.872821 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.873019 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.873114 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.873338 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.873423 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.873653 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.873750 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.873954 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.874038 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.874259 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.874344 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.874588 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.874687 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.874905 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.874991 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.875215 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.875350 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.875521 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.875615 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.875824 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.875915 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.876130 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.876215 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.876434 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.876519 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.876767 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.876860 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.877076 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.877161 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.877392 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.877476 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.877700 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.877799 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.878003 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.878092 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.878317 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.878407 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.878654 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.878768 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.878956 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.879048 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.879265 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.879348 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.879566 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.879660 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.879864 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.879949 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.880162 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.880254 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.880465 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.880550 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.880763 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.880853 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.881069 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.881153 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.881375 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.881461 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.881678 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.881763 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.882006 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.882138 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.882307 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.882399 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.882643 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.882737 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.882947 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.883032 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.883251 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.883336 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.883703 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.884341 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.884524 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.884789 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.884878 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.885117 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.885315 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.885440 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.885704 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.885796 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.886010 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.886282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.886367 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.886637 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.886786 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.886963 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.887246 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.887337 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.887580 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.887725 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.887882 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.888176 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.888262 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.888480 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.888642 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.888784 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.889084 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.889172 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.889390 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.889615 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.889711 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.889981 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.890072 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.890300 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.890536 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.890635 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.890933 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.891045 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.891256 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.891518 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.891604 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.891847 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.891968 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.892156 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.892466 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.892558 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.892773 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.893018 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.893112 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.893376 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.893495 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.893693 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.893957 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.894054 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.894292 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.894435 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.894685 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.894942 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.895039 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.895306 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.895419 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.895626 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.895924 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.896008 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.896247 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.896384 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.896547 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.896839 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.896924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.897142 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.897302 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.897441 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.897741 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.897825 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.898046 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.898215 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.898344 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.898666 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.898770 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.898965 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.899166 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.899289 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.899588 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.899686 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.899905 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.900029 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.900208 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.900484 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.900569 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.900791 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.900979 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.901094 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.901436 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.901534 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.901734 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.901895 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.902070 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.902343 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.902440 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.902729 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.902875 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.903080 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.903344 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.903563 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.903656 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.903803 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.903998 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.904246 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.904332 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.904552 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.904761 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.904847 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.905159 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.905267 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.905465 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.905710 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.905802 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.906075 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.906168 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.906392 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.906646 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.906734 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.907025 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.907110 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.907324 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.907522 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.907635 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.907971 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.908095 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.908302 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.908451 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.908613 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.908973 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.909073 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.909221 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.909381 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.909529 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.909834 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.909918 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.910140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.910297 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.910440 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.910763 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.910852 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.911081 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.911251 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.911384 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.911695 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.911786 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.912000 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.912181 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.912315 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.912618 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.912711 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.912930 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.913174 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.913268 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.913522 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.913646 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.913842 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.914120 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.914210 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.914494 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.914614 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.914813 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.915074 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.915159 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.915425 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.915549 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.915730 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.916034 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.916118 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.916333 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.916501 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.916632 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.916934 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.917026 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.917241 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.917398 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.917545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.917848 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.918010 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.918146 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.918364 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.918450 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.918767 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.918859 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.919065 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.919285 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.919372 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.919655 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.919746 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.919968 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.920174 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.920258 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.920574 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.920658 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.920889 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.921029 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.921195 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.921493 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.921588 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.921806 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.921978 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.922108 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.922406 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.922550 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.922729 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.922940 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.923063 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.923355 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.923450 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.923661 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.923843 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.923959 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.924270 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.924362 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.924585 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.924714 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.924888 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.925161 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.925245 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.925490 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.925633 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.925791 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.926104 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.926226 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.926409 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.926631 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.926718 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.927028 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.927111 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.927334 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.927508 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.927638 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.927943 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.928035 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.928259 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.928406 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.928564 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.928871 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.928952 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.929173 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.929363 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.929471 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.929775 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.929866 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.930088 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.930219 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.931228 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.931949 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.932374 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.932484 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.932570 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.932570 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.933401 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.933574 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.933731 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.934040 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.934125 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.934371 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.934529 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.934705 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.934989 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.935086 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.935300 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.935441 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.935602 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.935902 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.935986 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.936207 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.936435 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.936591 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.936794 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.936891 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.937092 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.937327 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.937412 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.937682 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.937766 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.937977 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.938238 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.938325 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.938581 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.938784 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.938894 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.939194 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.939293 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.939499 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.939672 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.939797 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.940112 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.940195 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.940415 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.940579 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.940711 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.941009 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.941093 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.941306 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.941479 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.941646 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.941925 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.942010 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.942267 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.942362 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.942611 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.942889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.942975 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.943237 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.943321 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.943537 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.943810 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.943895 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.944138 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.944253 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.944436 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.944711 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.944804 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.945023 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.945183 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.945326 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.945623 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.945754 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.945930 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.946684 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.946897 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.947162 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.947257 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.947496 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.947588 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.947794 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.948038 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.948126 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.948384 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.948469 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.948693 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.948972 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.949057 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.949276 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.949572 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.949759 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.949878 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.950181 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.950265 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.950497 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.950726 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.950812 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.951091 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.951174 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.951400 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.951639 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.951810 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.951988 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.952171 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.952286 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.952590 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.952717 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.952892 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.953135 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.953228 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.953491 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.953576 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.953794 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.954052 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.954138 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.954375 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.954522 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.954699 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.954979 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.955072 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.955282 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.955430 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.955584 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.955894 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.955985 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.956202 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.956337 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.956501 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.956776 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.956857 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.957088 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.957289 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.957373 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.957676 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.957759 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.957979 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.958139 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.958276 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.958596 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.958695 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.958893 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.959167 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.959265 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.959474 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.959623 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.959788 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.960085 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.960177 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.960380 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.960542 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.960679 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.960975 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.961058 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.961311 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.961396 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.961633 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.961888 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.961973 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.962274 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.962370 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.962601 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.962819 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.962906 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.963210 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.963293 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.963514 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.963679 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.963820 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.964117 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.964200 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.964425 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.964637 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.964730 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.965019 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.965103 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.965316 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.965552 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.965647 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.965913 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.966067 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.966215 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.966541 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.966640 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.966856 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.966999 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.967170 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.967485 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.967568 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.967782 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.967967 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.968084 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.968386 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.968476 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.968688 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.968828 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.968988 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.969292 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.969376 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.969595 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.969765 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.969899 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.970202 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.970293 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.970527 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.970695 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.970832 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.971133 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.971217 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.971442 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.971590 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.971756 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.972059 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.972215 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.972354 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.972658 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.972747 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.972951 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.973188 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.973281 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.973540 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.973623 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.973843 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.974102 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.974194 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.974559 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.974661 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.974872 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.975010 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.975176 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.975445 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.975530 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.975766 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.975888 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.976070 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.976340 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.976426 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.976674 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.976759 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.976970 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.977256 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.977417 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.977697 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.977783 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.978017 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.978150 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.978321 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.978616 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.978706 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.978928 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.979055 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.979230 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.979526 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.979613 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.979832 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.979976 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.980130 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.980416 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.980510 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.980730 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.980849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.981028 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.981313 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.981398 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.981726 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.981811 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.982032 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.982166 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.982331 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.982655 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.982744 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.983053 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.983195 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.983358 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.983571 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.983734 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.983972 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.984056 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.984291 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.984498 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.984591 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.984884 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.984968 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.985188 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.985382 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.985503 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.985811 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.985905 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.986112 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.986299 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.986421 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.986744 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.986839 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.987061 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.987175 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.987375 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.987651 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.987737 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.987984 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.988098 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.988390 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.988523 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.988707 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.988957 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.989049 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.989310 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.989451 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.989628 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.989888 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.989980 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.990214 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.990307 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.990538 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.990826 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.990912 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.991124 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.991284 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.991443 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.991720 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.991812 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.992030 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.992162 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.992333 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.992627 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.992719 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.993509 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.993732 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.994024 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.994135 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.994564 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.994700 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.994993 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.995119 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.995316 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.995576 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.995662 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.995945 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.996039 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.996266 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.996485 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.996570 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.996873 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.996957 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.997180 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.997352 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.997480 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.997769 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.997856 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.998066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.998255 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.998405 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.998725 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.998811 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.999041 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.999180 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.999368 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.999657 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.999749 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:08.999987 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.000094 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.000317 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.000563 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.000700 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.000942 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.001029 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.001257 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.001499 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.001624 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.001884 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.001978 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.002186 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.002360 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.002505 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.002817 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.002901 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.003121 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.003287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.003423 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.003728 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.003826 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.004033 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.004236 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.004323 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.004619 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.004735 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.004917 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.005195 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.005292 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.005521 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.005656 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.005838 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.006090 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.006189 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.006430 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.006603 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.006750 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.007047 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.007131 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.007382 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.007467 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.007682 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.007955 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.008040 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.008302 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.008395 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.009183 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.009489 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.009632 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.009800 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.010004 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.010099 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.010409 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.010530 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.010747 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.010923 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.011062 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.011364 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.011450 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.011667 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.011810 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.011975 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.012274 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.012360 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.012588 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.012710 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.013116 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.013199 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.013454 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.013546 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.013764 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.014072 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.014155 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.014380 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.014646 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.014735 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.015007 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.015110 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.015313 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.015554 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.015641 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.015912 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.016000 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.016216 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.016444 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.016537 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.016841 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.016932 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.017161 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.017318 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.017488 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.017805 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.017899 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.018116 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.018424 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.018554 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.018758 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.018923 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.019114 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.019369 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.019461 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.019713 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.019887 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.020025 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.020336 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.020421 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.020643 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.020782 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.020954 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.021243 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.021327 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.021556 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.021719 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.021863 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.022163 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.022255 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.022494 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.022679 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.022794 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.023101 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.023187 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.023422 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.023562 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.023748 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.024049 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.024145 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.024384 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.024545 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.024693 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.025004 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.025192 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.025517 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.025603 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.025833 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.026026 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.026166 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.026515 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.026682 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.026841 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.027126 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.027211 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.027460 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.027561 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.027786 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.028035 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.028131 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.028376 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.028493 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.028683 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.028961 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.029053 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.029274 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.029424 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.029591 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.029899 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.029983 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.030202 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.030373 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.030536 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.030852 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.030939 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.031278 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.031362 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.031706 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.031803 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.031995 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.032163 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.032313 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.032600 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.032688 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.032932 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.033060 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.033235 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.033520 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.033604 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.033843 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.033969 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.034144 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.034504 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.034593 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.034788 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.035089 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.035190 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.035390 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.035572 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.035697 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.036002 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.036084 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.036304 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.036510 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.036594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.036895 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.037015 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.037215 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.037437 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.037522 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.037802 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.037886 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.038101 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.038345 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.038430 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.038721 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.038818 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.039023 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.039247 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.039341 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.039609 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.039693 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.040443 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.040693 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.040778 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.041044 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.041137 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.041344 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.041573 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.041664 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.041988 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.042080 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.042292 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.042487 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.042614 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.042921 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.043022 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.043223 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.043392 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.043522 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.043823 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.043907 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.044130 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.044337 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.044429 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.044739 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.044823 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.045044 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.045277 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.045371 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.045686 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.045769 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.046099 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.046213 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.046570 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.046673 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.046889 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.047059 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.047327 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.047520 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.047659 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.047954 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.048038 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.048254 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.048453 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.048550 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.048862 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.048947 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.049167 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.049352 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.049474 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.049779 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.049864 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.050082 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.050265 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.050385 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.050719 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.050818 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.051025 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.051233 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.051317 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.051620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.051716 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.051924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.052140 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.052232 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.052517 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.052679 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.052818 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.053120 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.053204 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.053427 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.053576 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.053759 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.054037 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.054123 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.054369 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.054451 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.054703 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.054935 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.055021 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.055307 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.055489 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.056167 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.056434 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.056528 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.056766 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.056912 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.057073 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.057397 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.057488 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.057704 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.057854 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.058005 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.058622 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.058981 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.059080 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.059303 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.059438 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.059626 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.059898 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.059983 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.060247 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.060339 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.060572 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.060804 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.060893 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.061193 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.061287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.061511 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.061647 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.061822 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.062092 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.062177 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.062483 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.062570 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.062809 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.063028 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.063133 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.063435 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.063521 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.063737 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.063914 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.064038 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.064344 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.064437 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.064658 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.064824 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.064963 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.065264 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.065355 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.065565 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.065802 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.065887 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.066155 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.066246 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.066470 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.066737 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.066823 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.067066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.067254 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.067360 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.067655 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.067750 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.067953 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.068148 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.068233 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.068534 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.068631 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.068834 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.069103 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.069187 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.069423 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.069564 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.069722 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.070020 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.070104 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.070316 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.070645 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.070848 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.070947 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.071287 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.071372 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.071612 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.071769 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.071923 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.072210 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.072294 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.072539 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.072637 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.072848 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.073073 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.073160 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.073456 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.073541 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.073771 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.073932 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.074084 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.074373 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.074491 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.074728 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.074843 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.075035 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.075317 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.075449 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.075636 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.075831 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.075941 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.076247 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.076332 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.076552 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.076743 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.076875 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.077196 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.077289 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.077498 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.077699 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.077817 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.078128 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.078221 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.078445 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.078640 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.078780 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.079086 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.079170 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.079412 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.079618 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.079712 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.080012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.080100 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.080317 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.080473 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.080628 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.080933 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.081018 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.081281 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.081424 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.081564 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.081870 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.081954 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.082181 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.082318 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.082514 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.082827 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.082913 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.083146 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.083289 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.083452 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.083736 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.083823 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.084081 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.084165 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.084422 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.084594 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.084729 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.085038 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.085130 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.085378 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.085504 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.085687 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.085987 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.086081 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.086286 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.086491 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.086591 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105592) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 00:48:09.086981 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.087376 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.087828 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.088221 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.088820 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.089264 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.089673 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.090075 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.090492 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.090894 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.091336 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.091733 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.092111 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.092711 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.092986 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.093369 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.093748 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.094145 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.094585 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.094973 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.095369 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.095813 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.096418 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.096838 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.097254 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.097658 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.098114 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.098544 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.098927 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.099377 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.099761 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.100139 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.100526 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.100908 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.101340 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.101721 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.102103 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.103043 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.103426 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.103814 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.104192 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.104592 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.104982 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.105371 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.105787 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.106192 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.106612 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.106995 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.107399 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.107780 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.108255 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.108723 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.109190 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.109586 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.109972 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.110403 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.110814 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.111193 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.111629 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.112009 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.112390 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.112787 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.113170 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.113568 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.113956 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.114339 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.114794 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.115187 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.115884 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.116309 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.116702 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.117114 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.117542 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.118487 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.119010 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.119445 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.119863 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.120249 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.120633 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.121014 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.121424 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.121826 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.122264 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.122632 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.123018 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.123400 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.123779 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.124160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.124542 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.124923 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.125305 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.125697 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.126206 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.126638 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.127028 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.127454 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.127846 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.128228 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.128609 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.129068 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.129558 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.129983 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.130394 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.130813 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.131195 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.132078 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.132513 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.132883 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.133276 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.133833 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.134235 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.134682 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.135067 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.135472 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.135857 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.136299 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.136770 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.137173 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.137620 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.138007 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.138392 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.138804 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.139265 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.139699 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.140138 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.140517 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.140897 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.141280 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.141660 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.142109 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.142542 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.142934 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.143321 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.143698 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.144079 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.144456 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.144836 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.145217 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.145610 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.146060 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.146443 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.146813 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.147126 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.147441 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.147758 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.148106 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.148445 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.148770 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.149101 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.149473 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.149800 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.150116 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.150442 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.150801 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.151129 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.151444 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.151786 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.152101 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.152465 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.152726 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.153039 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.153354 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.153667 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.154009 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.154340 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.154683 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.155013 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.155333 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.155668 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.155989 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.156302 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.156637 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.156954 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.157268 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.157582 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.157895 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.158206 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.158555 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.158871 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.159184 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.159503 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.159823 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.160171 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.160501 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.160813 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.161143 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.161473 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.161833 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.162150 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.162529 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.162864 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.163180 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.163493 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.163802 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:48:09.164047 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105591) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 00:48:09.240481 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO dbcounter [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Registered counter for database nova_api Sep 18 00:48:09.262614 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_db.sqlalchemy.engines [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105592) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.269574 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writer thread running {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.317834 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO dbcounter [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Registered counter for database nova_api Sep 18 00:48:09.327650 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.context [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e1feb29a-fb47-481b-b99a-721bee8616c1(cell1) {{(pid=105592) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 00:48:09.331562 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:48:09.332028 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:48:09.333577 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:48:09.334291 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:48:09.334747 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:48:09.337337 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:48:09.339256 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_db.sqlalchemy.engines [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105591) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.339465 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO dbcounter [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Registered counter for database nova_cell0 Sep 18 00:48:09.343528 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO dbcounter [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Registered counter for database nova_cell1 Sep 18 00:48:09.345897 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writer thread running {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.350741 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_db.sqlalchemy.engines [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105592) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.352401 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_db.sqlalchemy.engines [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105592) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.358265 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writer thread running {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.360549 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writer thread running {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.403126 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.context [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e1feb29a-fb47-481b-b99a-721bee8616c1(cell1) {{(pid=105591) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 00:48:09.407070 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_concurrency.lockutils [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105591) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:48:09.407541 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_concurrency.lockutils [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105591) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:48:09.409099 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_concurrency.lockutils [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105591) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:48:09.409818 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_concurrency.lockutils [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105591) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:48:09.410263 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_concurrency.lockutils [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105591) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:48:09.412285 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_concurrency.lockutils [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105591) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:48:09.414301 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO dbcounter [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Registered counter for database nova_cell0 Sep 18 00:48:09.418626 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO dbcounter [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Registered counter for database nova_cell1 Sep 18 00:48:09.427986 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_db.sqlalchemy.engines [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105591) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.429399 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_db.sqlalchemy.engines [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105591) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.436235 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writer thread running {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.438965 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writer thread running {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.503943 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO dbcounter [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Registered counter for database nova_cell0 Sep 18 00:48:09.513729 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_db.sqlalchemy.engines [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105592) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.518287 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writer thread running {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.583239 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO dbcounter [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Registered counter for database nova_cell0 Sep 18 00:48:09.592862 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG oslo_db.sqlalchemy.engines [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105591) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:48:09.597567 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writer thread running {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:48:09.855752 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: WARNING keystonemiddleware.auth_token [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:48:09.863039 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: WARNING keystonemiddleware.auth_token [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:48:09.916475 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: WARNING keystonemiddleware.auth_token [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:48:09.923980 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: WARNING keystonemiddleware.auth_token [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:48:10.203377 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: WARNING keystonemiddleware.auth_token [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:48:10.209379 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: WARNING keystonemiddleware.auth_token [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:48:10.215679 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557d59a6b9c0 pid: 105592 (default app) Sep 18 00:48:10.217841 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:48:10.219047 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] 10.222.0.42 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001835 Sep 18 00:48:10.219451 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 788 bytes} [Sun Sep 18 00:48:10 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:48:10.267113 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: WARNING keystonemiddleware.auth_token [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:48:10.273209 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: WARNING keystonemiddleware.auth_token [None req-30cf6f9d-f765-4b8a-961d-73c1b34d43eb None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:48:10.278531 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557d59a6b9c0 pid: 105591 (default app) Sep 18 00:49:38.762438 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:49:38.763365 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001516 Sep 18 00:49:38.763774 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 2/2] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:49:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:49:38.776981 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_api:SELECT=3 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:38.777970 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_cell1:SELECT=3 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:38.779226 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_cell0:SELECT=3 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:38.780107 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_cell0:SELECT=3 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:39.919487 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-3aa1cccc-f020-456d-9dd0-9d92705c1584 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:49:39.960849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-3aa1cccc-f020-456d-9dd0-9d92705c1584 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.189814 Sep 18 00:49:39.962497 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 1/3] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 18 00:49:38 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:49:43.319899 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:49:43.321635 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-09f1056e-93c7-48d8-881e-4b38878335eb None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002378 Sep 18 00:49:43.322356 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 3/4] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:49:43 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:49:43.792757 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-879115f7-fd4a-4edd-8cc6-b04e6617cca0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "c1", "disk": 1, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "cirros256", "swap": 0, "ram": 256, "vcpus": 1, "os-flavor-access:is_public": true}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:43.843119 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-879115f7-fd4a-4edd-8cc6-b04e6617cca0 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.510026 Sep 18 00:49:43.846656 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 2/5] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:49:43 2022] POST /compute/v2.1/flavors => generated 407 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:49:44.070372 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-ef0ca519-a24c-4cf8-94f6-3eebee858fa4 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:44.076912 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_api:SELECT=3 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:44.078152 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_cell0:SELECT=3 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:44.078957 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_cell1:SELECT=3 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:44.079659 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:49:44.158804 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-ef0ca519-a24c-4cf8-94f6-3eebee858fa4 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.306340 Sep 18 00:49:44.159419 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 4/6] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 18 00:49:43 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:49:46.892851 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-879115f7-fd4a-4edd-8cc6-b04e6617cca0 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:49:46.893869 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-879115f7-fd4a-4edd-8cc6-b04e6617cca0 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001534 Sep 18 00:49:46.894527 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 3/7] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:49:46 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:49:47.747124 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-5855a8ac-e28b-42f1-9f9d-b121c806b032 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds512M", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "swap": 0, "id": "d1", "rxtx_factor": 1.0, "ram": 512, "disk": 5, "vcpus": 1}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:47.791710 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-5855a8ac-e28b-42f1-9f9d-b121c806b032 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.890628 Sep 18 00:49:47.796665 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 5/8] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:49:46 2022] POST /compute/v2.1/flavors => generated 404 bytes in 892 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:49:47.834517 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-1f811fdc-331e-45b3-88b1-a64c2d889294 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:47.906789 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-1f811fdc-331e-45b3-88b1-a64c2d889294 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.092426 Sep 18 00:49:47.908995 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 4/9] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 18 00:49:47 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:49:50.837504 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-5855a8ac-e28b-42f1-9f9d-b121c806b032 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:49:50.838645 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-5855a8ac-e28b-42f1-9f9d-b121c806b032 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001475 Sep 18 00:49:50.838938 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 6/10] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:49:50 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:49:51.367373 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-36d37c63-07a0-4ab5-b987-c3c991f3bd04 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "d2", "vcpus": 1, "name": "ds1G", "disk": 10}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:51.393275 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-36d37c63-07a0-4ab5-b987-c3c991f3bd04 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.546787 Sep 18 00:49:51.394019 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 5/11] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:49:50 2022] POST /compute/v2.1/flavors => generated 404 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 00:49:51.409938 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-6a9c807e-6fa8-4b3f-9bd8-bf609ee4a772 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:51.449355 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-6a9c807e-6fa8-4b3f-9bd8-bf609ee4a772 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.044028 Sep 18 00:49:51.450853 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 7/12] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 18 00:49:51 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:49:54.633980 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-36d37c63-07a0-4ab5-b987-c3c991f3bd04 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:49:54.634943 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-36d37c63-07a0-4ab5-b987-c3c991f3bd04 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Sep 18 00:49:54.635336 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 6/13] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:49:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 00:49:55.149354 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-db1a443f-1389-4538-8f7d-5493dbda8503 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "os-flavor-access:is_public": true, "name": "ds2G", "vcpus": 2, "disk": 10, "swap": 0, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:55.182615 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-db1a443f-1389-4538-8f7d-5493dbda8503 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.540446 Sep 18 00:49:55.183522 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 8/14] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:49:54 2022] POST /compute/v2.1/flavors => generated 404 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:49:55.196684 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-d3085928-2ff6-4fde-8550-5911d047c0e1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:55.227599 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-d3085928-2ff6-4fde-8550-5911d047c0e1 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035283 Sep 18 00:49:55.228404 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 7/15] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 18 00:49:55 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:49:58.113475 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-db1a443f-1389-4538-8f7d-5493dbda8503 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:49:58.114451 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-db1a443f-1389-4538-8f7d-5493dbda8503 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001510 Sep 18 00:49:58.114896 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 9/16] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:49:58 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:49:58.577104 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-872cbc32-88e3-4330-a23f-fa0c62499791 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "vcpus": 4, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds4G", "os-flavor-access:is_public": true, "id": "d4", "rxtx_factor": 1.0, "ram": 4096}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:58.601184 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-872cbc32-88e3-4330-a23f-fa0c62499791 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.479178 Sep 18 00:49:58.601906 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 8/17] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:49:58 2022] POST /compute/v2.1/flavors => generated 404 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:49:58.618122 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-457053ad-ced5-4bbf-b1a2-f396ac393158 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:49:58.652721 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-457053ad-ced5-4bbf-b1a2-f396ac393158 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039916 Sep 18 00:49:58.653458 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 10/18] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 18 00:49:58 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:50:01.431473 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-872cbc32-88e3-4330-a23f-fa0c62499791 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:01.432607 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-872cbc32-88e3-4330-a23f-fa0c62499791 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001590 Sep 18 00:50:01.433016 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 9/19] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:50:01 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:01.960891 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-3a71ea6c-79f8-4df9-8844-48c7c24e86d2 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:01.993614 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-3a71ea6c-79f8-4df9-8844-48c7c24e86d2 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.553780 Sep 18 00:50:01.994162 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 11/20] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 18 00:50:01 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:50:04.778614 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-872cbc32-88e3-4330-a23f-fa0c62499791 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:04.779863 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-872cbc32-88e3-4330-a23f-fa0c62499791 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001752 Sep 18 00:50:04.780411 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 10/21] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:50:04 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 00:50:05.283610 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-c9f810f2-b6f6-4388-969c-f90884655299 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 512, "name": "m1.tiny", "swap": 0, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "1"}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:05.311343 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-c9f810f2-b6f6-4388-969c-f90884655299 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.523627 Sep 18 00:50:05.314253 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 12/22] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:50:04 2022] POST /compute/v2.1/flavors => generated 402 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:05.326322 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-7839a3f9-d4f5-46c4-8762-1b8c590792e2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:05.356929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-7839a3f9-d4f5-46c4-8762-1b8c590792e2 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034823 Sep 18 00:50:05.357529 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 11/23] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 18 00:50:05 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:50:08.081452 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-c9f810f2-b6f6-4388-969c-f90884655299 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:08.083966 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-c9f810f2-b6f6-4388-969c-f90884655299 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003261 Sep 18 00:50:08.084873 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:50:08 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:08.717232 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-e1930c24-7097-4379-b363-b77fa1714fa1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "swap": 0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.small", "id": "2", "disk": 20, "vcpus": 1, "rxtx_factor": 1.0}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:08.741547 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-e1930c24-7097-4379-b363-b77fa1714fa1 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.644996 Sep 18 00:50:08.742268 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 12/25] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:50:08 2022] POST /compute/v2.1/flavors => generated 405 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:08.763379 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-b9579513-088a-4227-9503-10eb5b320eb6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:08.797388 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-b9579513-088a-4227-9503-10eb5b320eb6 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040231 Sep 18 00:50:08.797986 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 14/26] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 18 00:50:08 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:50:11.791193 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-e1930c24-7097-4379-b363-b77fa1714fa1 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:11.794095 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-e1930c24-7097-4379-b363-b77fa1714fa1 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003788 Sep 18 00:50:11.795625 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 13/27] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:50:11 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:12.391873 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-77fa8b8e-83fd-473a-8a8a-7c49b2447f63 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "swap": 0, "id": "3", "ram": 4096, "os-flavor-access:is_public": true, "disk": 40, "name": "m1.medium"}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:12.427561 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-77fa8b8e-83fd-473a-8a8a-7c49b2447f63 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.625466 Sep 18 00:50:12.428642 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 15/28] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:50:11 2022] POST /compute/v2.1/flavors => generated 406 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:12.447197 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-5dd46418-5455-475a-85d0-384ba51fec43 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:12.452568 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:12.488218 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-5dd46418-5455-475a-85d0-384ba51fec43 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.045535 Sep 18 00:50:12.503577 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 14/29] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 18 00:50:12 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:50:15.692978 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-77fa8b8e-83fd-473a-8a8a-7c49b2447f63 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:15.694545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-77fa8b8e-83fd-473a-8a8a-7c49b2447f63 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001978 Sep 18 00:50:15.694954 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 16/30] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:50:15 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:16.320930 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-f8d5d7bc-21a0-4561-87f8-41b6a713c6fd admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "swap": 0, "name": "m1.large", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "4", "vcpus": 4, "disk": 80, "rxtx_factor": 1.0, "ram": 8192}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:16.343162 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-f8d5d7bc-21a0-4561-87f8-41b6a713c6fd admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.640797 Sep 18 00:50:16.346506 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 15/31] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:50:15 2022] POST /compute/v2.1/flavors => generated 405 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 18 00:50:16.359098 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-376e43a8-d064-46e2-b48e-d62a739c9161 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:16.364210 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_api:SELECT=39,nova_api:SAVEPOINT=4,nova_api:INSERT=8,nova_api:RELEASE=4 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:16.395436 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-376e43a8-d064-46e2-b48e-d62a739c9161 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040226 Sep 18 00:50:16.396134 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 17/32] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 18 00:50:16 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:50:19.176235 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-f8d5d7bc-21a0-4561-87f8-41b6a713c6fd admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:19.177163 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-f8d5d7bc-21a0-4561-87f8-41b6a713c6fd admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001502 Sep 18 00:50:19.177593 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 16/33] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:50:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:19.653416 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-5378b603-ecfd-477e-8916-5642f9315e23 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "5", "disk": 160, "name": "m1.xlarge", "vcpus": 8, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:19.683370 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-5378b603-ecfd-477e-8916-5642f9315e23 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.496210 Sep 18 00:50:19.684121 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 18/34] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 18 00:50:19 2022] POST /compute/v2.1/flavors => generated 408 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:19.696474 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-46ae6acc-1cd5-4d8e-8bb2-d93dfeab896d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:19.731786 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-46ae6acc-1cd5-4d8e-8bb2-d93dfeab896d admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039160 Sep 18 00:50:19.732569 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 17/35] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 18 00:50:19 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:50:34.947907 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-5378b603-ecfd-477e-8916-5642f9315e23 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:34.948922 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-5378b603-ecfd-477e-8916-5642f9315e23 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Sep 18 00:50:34.949334 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 19/36] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:50:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:34.959775 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:35.346296 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-58e81dff-233e-4492-817e-f83b3b757a8f admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:35.365657 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-58e81dff-233e-4492-817e-f83b3b757a8f admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.410252 Sep 18 00:50:35.366193 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 18/37] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 18 00:50:34 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:50:36.723969 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-5378b603-ecfd-477e-8916-5642f9315e23 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:36.724980 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-5378b603-ecfd-477e-8916-5642f9315e23 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001431 Sep 18 00:50:36.725381 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 20/38] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:37.255348 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-007a92a6-dea1-4a56-bb2b-6338214f653b admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:37.261022 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.wsgi [None req-007a92a6-dea1-4a56-bb2b-6338214f653b admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-007a92a6-dea1-4a56-bb2b-6338214f653b admin admin] Request method failure captured: Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Accept: application/json Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Accept-Encoding: gzip, deflate Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Connection: keep-alive Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Content-Length: 0 Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Host: 10.222.0.42 Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Openstack-Api-Version: compute 2.92 Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Openstack-System-Scope: None Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: User-Agent: python-novaclient Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Auth-Token: gAAAAABjJmtcGLNUkEvwewd9I-OhJVR_Z_5hUfCIj9huXS_nTXU-sCS2tGyXmkCV2ik4SRBto-Xk0v7EMOVHbdCNXqg7htS713ykXkRFrPuD1liV15mU0qCWQ08Z6MvSMV5cHPAY2lf-dIALCYfuZ16pN9OAKYgjn1JVA_Jb0eO44ALbcu7N_eU Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Domain-Id: None Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Domain-Name: None Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Identity-Status: Confirmed Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Is-Admin-Project: True Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Domain-Id: default Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Domain-Name: Default Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Name: admin Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Role: member,admin,reader Sep 18 00:50:37.264282 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Roles: member,admin,reader Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}] Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant: admin Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant-Name: admin Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User: admin Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Domain-Id: default Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Domain-Name: Default Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Id: 6f5558a706da458791737ed054e51874 Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Name: admin Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: method: > Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: exception: Flavor 42 could not be found. Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: action_args: {'id': '42'} Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:50:37.265929 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:37.266959 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.268287 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-007a92a6-dea1-4a56-bb2b-6338214f653b admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105591) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:50:37.269532 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-007a92a6-dea1-4a56-bb2b-6338214f653b admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.404326 Sep 18 00:50:37.269532 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 19/39] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 18 00:50:36 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 405 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:37.275542 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-2099518d-65c6-466b-9eab-5b298e4557ba admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:37.279701 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:INSERT=2,nova_api:RELEASE=1 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:50:37.299947 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-2099518d-65c6-466b-9eab-5b298e4557ba admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027951 Sep 18 00:50:37.300518 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 21/40] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 18 00:50:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:50:37.312121 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-56c0bcd2-cfdd-4bae-953e-57005f94f414 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:37.334708 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-56c0bcd2-cfdd-4bae-953e-57005f94f414 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027165 Sep 18 00:50:37.335253 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 20/41] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 18 00:50:37 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:50:37.345549 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-af229ff1-c515-48e0-8e29-fb446258b736 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:37.351399 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.wsgi [None req-af229ff1-c515-48e0-8e29-fb446258b736 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-af229ff1-c515-48e0-8e29-fb446258b736 admin admin] Request method failure captured: Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Accept: application/json Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Accept-Encoding: gzip, deflate Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Connection: keep-alive Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Content-Length: 0 Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Host: 10.222.0.42 Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Openstack-Api-Version: compute 2.92 Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Openstack-System-Scope: None Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: User-Agent: python-novaclient Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Auth-Token: gAAAAABjJmtcGLNUkEvwewd9I-OhJVR_Z_5hUfCIj9huXS_nTXU-sCS2tGyXmkCV2ik4SRBto-Xk0v7EMOVHbdCNXqg7htS713ykXkRFrPuD1liV15mU0qCWQ08Z6MvSMV5cHPAY2lf-dIALCYfuZ16pN9OAKYgjn1JVA_Jb0eO44ALbcu7N_eU Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Domain-Id: None Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Domain-Name: None Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Identity-Status: Confirmed Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Is-Admin-Project: True Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Domain-Id: default Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Domain-Name: Default Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Name: admin Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Role: member,admin,reader Sep 18 00:50:37.354696 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Roles: member,admin,reader Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}] Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant: admin Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant-Name: admin Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User: admin Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Domain-Id: default Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Domain-Name: Default Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Id: 6f5558a706da458791737ed054e51874 Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Name: admin Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: method: > Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: exception: Flavor 42 could not be found. Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: action_args: {'id': '42'} Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:50:37.355924 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:37.356716 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:37.358095 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-af229ff1-c515-48e0-8e29-fb446258b736 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105592) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:50:37.359488 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-af229ff1-c515-48e0-8e29-fb446258b736 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014894 Sep 18 00:50:37.359488 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 22/42] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 18 00:50:37 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:38.608065 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-56c0bcd2-cfdd-4bae-953e-57005f94f414 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:38.609229 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-56c0bcd2-cfdd-4bae-953e-57005f94f414 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001633 Sep 18 00:50:38.609690 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 21/43] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:38 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:39.172931 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-d9ad0051-2a17-4c07-9f17-743dd3f92145 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:39.178121 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.wsgi [None req-d9ad0051-2a17-4c07-9f17-743dd3f92145 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-d9ad0051-2a17-4c07-9f17-743dd3f92145 admin admin] Request method failure captured: Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Accept: application/json Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Accept-Encoding: gzip, deflate Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Connection: keep-alive Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Content-Length: 0 Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Host: 10.222.0.42 Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Openstack-Api-Version: compute 2.92 Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Openstack-System-Scope: None Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: User-Agent: python-novaclient Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Auth-Token: gAAAAABjJmteoiVd2RrJ3eAJqCEQ8JNW4THXMKNLiU2Xxl6JVD8DcyXwapDngi-_kTeF_s00zL47YFuYDlMiqvPmnALCoAbBFr1YoSts4ad-2y_3Wbp9q2aRDNVgvmJUrnDNrrWIZZ4rKQuNkc4oEP_JScWC3nQs6jHO31mhU6fESiz4hkzqvd0 Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Domain-Id: None Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Domain-Name: None Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Identity-Status: Confirmed Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Is-Admin-Project: True Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Domain-Id: default Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Domain-Name: Default Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Name: admin Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Role: member,admin,reader Sep 18 00:50:39.179620 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Roles: member,admin,reader Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}] Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant: admin Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant-Name: admin Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User: admin Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Domain-Id: default Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Domain-Name: Default Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Id: 6f5558a706da458791737ed054e51874 Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Name: admin Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: method: > Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: exception: Flavor 84 could not be found. Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: action_args: {'id': '84'} Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:50:39.181066 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:39.182720 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.184140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-d9ad0051-2a17-4c07-9f17-743dd3f92145 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105592) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:50:39.185551 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-d9ad0051-2a17-4c07-9f17-743dd3f92145 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.434211 Sep 18 00:50:39.185551 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 23/44] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 18 00:50:38 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 435 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:39.191288 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-a6a3f739-3475-418e-9521-61c902579d28 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:39.206961 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-a6a3f739-3475-418e-9521-61c902579d28 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019239 Sep 18 00:50:39.207515 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 22/45] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 18 00:50:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:50:39.217675 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-68e89743-9c4e-4474-be11-11b4d746814a admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:39.233901 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-68e89743-9c4e-4474-be11-11b4d746814a admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019711 Sep 18 00:50:39.234428 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 18 00:50:39 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:50:39.243945 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-9c054ba5-6af6-40e5-8b81-7f75acca72a9 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:39.249008 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.wsgi [None req-9c054ba5-6af6-40e5-8b81-7f75acca72a9 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-9c054ba5-6af6-40e5-8b81-7f75acca72a9 admin admin] Request method failure captured: Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Accept: application/json Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Accept-Encoding: gzip, deflate Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Connection: keep-alive Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Content-Length: 0 Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Host: 10.222.0.42 Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Openstack-Api-Version: compute 2.92 Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Openstack-System-Scope: None Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: User-Agent: python-novaclient Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Auth-Token: gAAAAABjJmteoiVd2RrJ3eAJqCEQ8JNW4THXMKNLiU2Xxl6JVD8DcyXwapDngi-_kTeF_s00zL47YFuYDlMiqvPmnALCoAbBFr1YoSts4ad-2y_3Wbp9q2aRDNVgvmJUrnDNrrWIZZ4rKQuNkc4oEP_JScWC3nQs6jHO31mhU6fESiz4hkzqvd0 Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Domain-Id: None Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Domain-Name: None Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Identity-Status: Confirmed Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Is-Admin-Project: True Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Domain-Id: default Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Domain-Name: Default Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Name: admin Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Role: member,admin,reader Sep 18 00:50:39.250840 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Roles: member,admin,reader Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}] Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant: admin Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant-Name: admin Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User: admin Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Domain-Id: default Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Domain-Name: Default Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Id: 6f5558a706da458791737ed054e51874 Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Name: admin Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: method: > Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: exception: Flavor 84 could not be found. Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: action_args: {'id': '84'} Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:50:39.252308 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:39.253088 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:39.254160 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-9c054ba5-6af6-40e5-8b81-7f75acca72a9 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105591) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:50:39.255205 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-9c054ba5-6af6-40e5-8b81-7f75acca72a9 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011824 Sep 18 00:50:39.255205 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 23/47] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 18 00:50:39 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:40.496871 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-68e89743-9c4e-4474-be11-11b4d746814a admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:40.497914 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-68e89743-9c4e-4474-be11-11b4d746814a admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Sep 18 00:50:40.498313 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 25/48] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:41.027470 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-7df8105f-04eb-4a77-b50f-89afacc7da87 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:41.033343 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.wsgi [None req-7df8105f-04eb-4a77-b50f-89afacc7da87 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-7df8105f-04eb-4a77-b50f-89afacc7da87 admin admin] Request method failure captured: Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Accept: application/json Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Accept-Encoding: gzip, deflate Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Connection: keep-alive Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Content-Length: 0 Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Host: 10.222.0.42 Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Openstack-Api-Version: compute 2.92 Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Openstack-System-Scope: None Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: User-Agent: python-novaclient Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Auth-Token: gAAAAABjJmtgba_fAYmcqviIUc0aT1tXreTvYcWBFi53SukaJumVldjl_xXG_1pAeveKIJd_tC6UOWfzhLaaJOlZ1Rf7OmKvnyK7hRpgzWW5X1KKABV9T8KGzQ65uj70o88DpvlayIzkUYXC0DjIYDqxguJq9PTO1swopuX9iuyDM3s0sWjRHO4 Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Domain-Id: None Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Domain-Name: None Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Identity-Status: Confirmed Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Is-Admin-Project: True Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Domain-Id: default Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Domain-Name: Default Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Project-Name: admin Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Role: member,admin,reader Sep 18 00:50:41.034889 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Roles: member,admin,reader Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}] Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant: admin Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-Tenant-Name: admin Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User: admin Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Domain-Id: default Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Domain-Name: Default Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Id: 6f5558a706da458791737ed054e51874 Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: X-User-Name: admin Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: method: > Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: exception: Flavor 451 could not be found. Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: action_args: {'id': '451'} Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:50:41.036455 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:41.037849 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.038975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-7df8105f-04eb-4a77-b50f-89afacc7da87 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105591) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:50:41.040169 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-7df8105f-04eb-4a77-b50f-89afacc7da87 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.397832 Sep 18 00:50:41.040169 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 24/49] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 18 00:50:40 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 400 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:41.046897 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-a76a97cf-275e-4e82-93b2-12411b334c0f admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:41.062940 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-a76a97cf-275e-4e82-93b2-12411b334c0f admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019571 Sep 18 00:50:41.063485 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 26/50] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 18 00:50:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:50:41.073015 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:41.088456 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018962 Sep 18 00:50:41.089000 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 25/51] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 18 00:50:41 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:50:41.098914 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-9eb01d12-9b31-4907-b398-2f18ee30e2f8 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:41.104956 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.wsgi [None req-9eb01d12-9b31-4907-b398-2f18ee30e2f8 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-9eb01d12-9b31-4907-b398-2f18ee30e2f8 admin admin] Request method failure captured: Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Accept: application/json Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Accept-Encoding: gzip, deflate Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Connection: keep-alive Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Content-Length: 0 Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Host: 10.222.0.42 Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Openstack-Api-Version: compute 2.92 Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Openstack-System-Scope: None Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: User-Agent: python-novaclient Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Auth-Token: gAAAAABjJmtgba_fAYmcqviIUc0aT1tXreTvYcWBFi53SukaJumVldjl_xXG_1pAeveKIJd_tC6UOWfzhLaaJOlZ1Rf7OmKvnyK7hRpgzWW5X1KKABV9T8KGzQ65uj70o88DpvlayIzkUYXC0DjIYDqxguJq9PTO1swopuX9iuyDM3s0sWjRHO4 Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Domain-Id: None Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Domain-Name: None Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Identity-Status: Confirmed Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Is-Admin-Project: True Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Openstack-Nova-Api-Version: 2.92 Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Domain-Id: default Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Domain-Name: Default Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Project-Name: admin Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Role: member,admin,reader Sep 18 00:50:41.106545 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Roles: member,admin,reader Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2e41e0644d514666af8ca7ed05ebaec8"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2e41e0644d514666af8ca7ed05ebaec8"}]}] Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant: admin Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant-Id: 2e41e0644d514666af8ca7ed05ebaec8 Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-Tenant-Name: admin Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User: admin Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Domain-Id: default Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Domain-Name: Default Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Id: 6f5558a706da458791737ed054e51874 Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: X-User-Name: admin Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: method: > Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: exception: Flavor 451 could not be found. Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: action_args: {'id': '451'} Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:50:41.108012 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:41.109184 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: ERROR nova.api.openstack.wsgi Sep 18 00:50:41.110594 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-9eb01d12-9b31-4907-b398-2f18ee30e2f8 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105592) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:50:41.111860 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-9eb01d12-9b31-4907-b398-2f18ee30e2f8 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012723 Sep 18 00:50:41.111860 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 27/52] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 18 00:50:41 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 18 00:50:42.353100 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:42.354098 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001439 Sep 18 00:50:42.354516 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:42.880420 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-dbf9c8d6-0028-4c8b-8c52-bf218b51535e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:42.901227 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-dbf9c8d6-0028-4c8b-8c52-bf218b51535e admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.407197 Sep 18 00:50:42.901786 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 28/54] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 18 00:50:42 2022] POST /compute/v2.1/flavors => generated 404 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:44.139905 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:44.140905 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001436 Sep 18 00:50:44.141303 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 27/55] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:44 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:44.689383 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-fa2ad0e1-5611-460a-8c8e-de6039f5e293 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:44.709019 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-fa2ad0e1-5611-460a-8c8e-de6039f5e293 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.426451 Sep 18 00:50:44.709596 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 29/56] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 18 00:50:44 2022] POST /compute/v2.1/flavors => generated 405 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:45.953385 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:45.954390 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Sep 18 00:50:45.954825 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 28/57] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:46.483857 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-5eadc1f8-6504-40cd-a877-25d375c92134 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:46.504499 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-5eadc1f8-6504-40cd-a877-25d375c92134 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.410845 Sep 18 00:50:46.505049 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 30/58] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 18 00:50:46 2022] POST /compute/v2.1/flavors => generated 407 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:47.733030 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:47.734059 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001437 Sep 18 00:50:47.734479 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 29/59] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:48.260842 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-95f174c3-139d-4a74-a3e4-cd230fec68fc admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:50:48.281146 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-95f174c3-139d-4a74-a3e4-cd230fec68fc admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.410176 Sep 18 00:50:48.281700 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 31/60] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 18 00:50:47 2022] POST /compute/v2.1/flavors => generated 409 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:50:49.530665 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:49.531673 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Sep 18 00:50:49.532075 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 30/61] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:50:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:50:50.068423 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-5632a8d4-56ef-4765-bbb5-f9ed087cdacd admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:50:50.089450 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-5632a8d4-56ef-4765-bbb5-f9ed087cdacd admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.415033 Sep 18 00:50:50.089994 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 32/62] 10.222.0.42 () {60 vars in 1202 bytes} [Sun Sep 18 00:50:49 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:51:47.687754 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:51:47.688760 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ae642871-1a8a-4387-8898-100b4dba02d6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001704 Sep 18 00:51:47.689158 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 31/63] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 18 00:51:47 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:51:47.700086 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:51:48.086683 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-2f797663-7060-48c6-ae33-6442122e9086 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:51:48.108140 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-2f797663-7060-48c6-ae33-6442122e9086 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.412387 Sep 18 00:51:48.108692 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 33/64] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 18 00:51:47 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:51:48.131101 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_api:SELECT=14 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:51:48.133688 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-ccc3dca9-96f6-4a9b-a0cb-49e8e4ee0a08 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:51:48.141277 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-ccc3dca9-96f6-4a9b-a0cb-49e8e4ee0a08 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012966 Sep 18 00:51:48.141827 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 32/65] 10.222.0.42 () {60 vars in 1283 bytes} [Sun Sep 18 00:51:48 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:51:48.154345 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-df93ece6-6f2f-4cf9-8c42-b8ac26797e87 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:51:48.162344 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-df93ece6-6f2f-4cf9-8c42-b8ac26797e87 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011751 Sep 18 00:51:48.162911 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 34/66] 10.222.0.42 () {60 vars in 1286 bytes} [Sun Sep 18 00:51:48 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:51:48.174340 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:51:48.181713 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010929 Sep 18 00:51:48.182276 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 33/67] 10.222.0.42 () {60 vars in 1286 bytes} [Sun Sep 18 00:51:48 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:51:48.193957 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-554e5418-1b96-4116-be03-569a116bc3ec admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:51:48.201233 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-554e5418-1b96-4116-be03-569a116bc3ec admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.010815 Sep 18 00:51:48.201750 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 35/68] 10.222.0.42 () {60 vars in 1283 bytes} [Sun Sep 18 00:51:48 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:54:46.723975 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:54:46.725015 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001487 Sep 18 00:54:46.725417 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 34/69] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:54:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:54:46.868964 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_api:SELECT=6 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:54:47.035975 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:54:47.041331 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:54:47.041795 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:54:47.042214 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:54:47.042940 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:54:47.043377 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:54:47.043790 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:54:47.075781 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-da7d4a81-cbb2-4f76-b640-7a780aba14e0 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.207967 Sep 18 00:54:47.076339 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 36/70] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 18 00:54:46 2022] GET /compute/v2.1/os-services => generated 791 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:54:48.180055 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:54:48.181072 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001430 Sep 18 00:54:48.181479 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 35/71] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:54:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:54:48.373015 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:54:48.376494 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:54:48.376994 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:54:48.377419 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:54:48.378127 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:54:48.378595 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:54:48.379011 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:54:48.400034 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-16018331-25da-4f57-8212-aa7ce2c40946 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.079603 Sep 18 00:54:48.400602 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 37/72] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 18 00:54:48 2022] GET /compute/v2.1/os-services => generated 791 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:54:59.891971 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:54:59.893002 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001471 Sep 18 00:54:59.893394 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 36/73] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:54:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:55:00.034674 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_cell0:SELECT=4 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:00.035391 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_cell1:SELECT=4 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:00.036225 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG dbcounter [-] [105592] Writing DB stats nova_api:SELECT=8 {{(pid=105592) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:55:00.086295 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:55:00.089860 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:55:00.090360 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:55:00.090799 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:55:00.091520 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:55:00.091962 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:55:00.092413 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:55:00.118431 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-e945d9c8-52c4-4371-b7cd-6e95b44415f5 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.088092 Sep 18 00:55:00.118958 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 38/74] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 18 00:55:00 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:55:01.219433 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG nova.api.openstack.wsgi [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] Calling method '>' {{(pid=105591) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:55:01.220470 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: INFO nova.api.openstack.requestlog [None req-5b3a1e8e-2c8b-4280-89c2-4e9310d93be6 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Sep 18 00:55:01.220871 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: [pid: 105591|app: 0|req: 37/75] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 18 00:55:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 00:55:01.412788 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG nova.api.openstack.wsgi [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] Calling method '>' {{(pid=105592) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:55:01.416164 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:55:01.416642 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:55:01.417102 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:55:01.417812 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:55:01.418254 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:55:01.418697 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: DEBUG oslo_concurrency.lockutils [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105592) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:55:01.440301 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: INFO nova.api.openstack.requestlog [None req-c29d24ed-9359-4f8c-9362-aabf780f58f0 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.079246 Sep 18 00:55:01.440800 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: [pid: 105592|app: 0|req: 39/76] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 18 00:55:01 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:56:00.960479 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: Sun Sep 18 00:56:00 2022 - graceful shutdown triggered... Sep 18 00:56:00.960612 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 18 00:56:00.961426 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: Gracefully killing worker 1 (pid: 105591)... Sep 18 00:56:00.961793 dv-cinder-master-3-iscsi devstack@n-api.service[105592]: Gracefully killing worker 2 (pid: 105592)... Sep 18 00:56:01.090212 dv-cinder-master-3-iscsi devstack@n-api.service[105591]: DEBUG dbcounter [-] [105591] Writing DB stats nova_api:SELECT=4 {{(pid=105591) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:01.962406 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: worker 1 buried after 1 seconds Sep 18 00:56:01.962980 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: worker 2 buried after 1 seconds Sep 18 00:56:01.962980 dv-cinder-master-3-iscsi devstack@n-api.service[105590]: goodbye to uWSGI. Sep 18 00:56:01.964032 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 18 00:56:01.964592 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 18 00:56:01.967387 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 18 00:56:01.977138 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 18 00:56:01.979812 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 18 00:56:01 2022] *** Sep 18 00:56:01.979971 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 18 00:56:01.980111 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 18 00:56:01.980233 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: nodename: dv-cinder-master-3-iscsi Sep 18 00:56:01.980335 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: machine: x86_64 Sep 18 00:56:01.980442 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: clock source: unix Sep 18 00:56:01.980550 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: pcre jit disabled Sep 18 00:56:01.980656 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: detected number of CPU cores: 4 Sep 18 00:56:01.980763 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: current working directory: / Sep 18 00:56:01.980877 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: detected binary path: /usr/bin/uwsgi-core Sep 18 00:56:01.980999 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: your processes number limit is 47680 Sep 18 00:56:01.981454 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: your memory page size is 4096 bytes Sep 18 00:56:01.981587 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: detected max file descriptor number: 2048 Sep 18 00:56:01.981699 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: lock engine: pthread robust mutexes Sep 18 00:56:01.981900 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: thunder lock: enabled Sep 18 00:56:01.982058 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 18 00:56:01.982183 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 18 00:56:02.022952 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: Python main interpreter initialized at 0x562e774de9c0 Sep 18 00:56:02.022952 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: python threads support enabled Sep 18 00:56:02.022952 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: your server socket listen backlog is limited to 100 connections Sep 18 00:56:02.022952 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: your mercy for graceful operations on workers is 90 seconds Sep 18 00:56:02.023470 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: mapped 403077 bytes (393 KB) for 2 cores Sep 18 00:56:02.023470 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: *** Operational MODE: preforking *** Sep 18 00:56:02.023769 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: *** uWSGI is running in multiple interpreter mode *** Sep 18 00:56:02.023769 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: spawned uWSGI master process (pid: 120813) Sep 18 00:56:02.023906 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 18 00:56:02.024627 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: spawned uWSGI worker 1 (pid: 120831, cores: 1) Sep 18 00:56:02.025143 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: spawned uWSGI worker 2 (pid: 120832, cores: 1) Sep 18 00:56:02.025445 dv-cinder-master-3-iscsi devstack@n-api.service[120813]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 18 00:56:02.694451 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 00:56:02.792815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 18 00:56:03.857250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:56:03.857250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: from cryptography.utils import int_from_bytes Sep 18 00:56:03.857907 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:56:03.857907 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: from cryptography.utils import int_from_bytes Sep 18 00:56:03.939392 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:56:03.939392 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: from cryptography.utils import int_from_bytes Sep 18 00:56:03.940104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 18 00:56:03.940104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: from cryptography.utils import int_from_bytes Sep 18 00:56:06.295483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 00:56:06.296100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 00:56:06.296390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 00:56:06.296650 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 00:56:06.296917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 00:56:06.297317 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.297649 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.297979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.298306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.298671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.298990 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.299319 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.299647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.299982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.300250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.300572 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.301040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.301371 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.301688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.302007 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.302333 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.302673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.303045 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.303361 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.303870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.305348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.305348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.305348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.305348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.305665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.305879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.306218 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.306562 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.306887 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.307207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.307607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.307938 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.308248 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.308568 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.308993 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.309311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.309625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.309954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.310310 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.310655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.310975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.311328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.311642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.311974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.312264 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.312569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.312888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.313211 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.313529 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.313841 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.314100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.314417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.314771 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.315090 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.315423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.315680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.315935 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.316283 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.316586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.316909 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.317216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.317532 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.317839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.318152 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.318487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.319589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.319903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.320233 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.320559 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.320889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.321203 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.321530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.321972 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.322199 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.322544 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.322899 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.323224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.323542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.323869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.324198 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.324561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.324827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.325141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.325467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.325807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.326124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.326383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.326720 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.327035 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.327366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.327680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.327993 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.328304 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.328616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.328929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.329241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.329560 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.329869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.330200 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.330517 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.330847 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.331146 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.331463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.331773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.332099 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.332430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.332744 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.333058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.333369 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.333682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.334013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.334982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.335249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.335569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.335884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.336699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.336699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.336881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.337114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.337443 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.337754 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.338069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.338390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.338735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.339046 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.339359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.339670 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.339982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.340295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.340606 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.340917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.341225 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.341546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.341874 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.342188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.342522 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.342838 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.343165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.343511 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.343849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.344199 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.344520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.344858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.345195 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.345521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.345839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.346162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.346507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.346831 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.347149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.347480 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.347798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.348125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.348463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.348787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.349073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 18 00:56:06.349516 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.349589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 18 00:56:06.349668 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.349745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.349921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 18 00:56:06.350070 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.350397 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.350759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.351080 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.351202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 18 00:56:06.351427 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.351784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.351905 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 18 00:56:06.352103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.352437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.352772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.352921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.353091 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.353411 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.353698 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.353808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.354070 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.354392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.354613 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.354733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.355057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.355342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.355464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.355693 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.356017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.356226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.356334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.356642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.356951 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.357061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.357269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.357624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.357825 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.357965 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.358265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.358624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.358949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.359277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.359595 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.359912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.360231 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.360473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.360640 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.360837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.361148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.361466 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.361677 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.361772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.362096 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.362310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.362422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.362761 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.363108 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.363337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.363453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.363732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.364051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.364236 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.364367 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.364682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.365004 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.365111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.365321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.365666 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.365895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.366000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.366303 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.366643 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.366938 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.367120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.367281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.367458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.367589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.367964 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.368177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.368322 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.368478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.368659 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.369131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.369274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.369758 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.369883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.370122 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.370209 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.370573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.370832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.370933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.371320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.371473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.371682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.372011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.372347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.372684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.373011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.373328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.373648 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.373963 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.374108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.374359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.374746 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.375042 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.375363 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.375656 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.376417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.376417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.376566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.376883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.377001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.377200 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.378048 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.378048 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.378217 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.378493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.378804 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.379159 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.379437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.379540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.379821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.380155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.380444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.380545 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.380784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.381612 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.381935 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.382059 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.382254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.382614 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.382735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.385536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.385663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.385761 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.385843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.385920 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.385990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.386064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.386316 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.386402 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.386512 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.386601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.386601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.386735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.386813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.386882 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.386958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.387101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.387260 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387345 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.387421 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387497 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.387571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.387707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387782 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.387858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.387988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.388062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.388144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.388214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.388282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.388896 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.389006 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.389086 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.389168 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.389489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.389641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.389828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.389828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.390120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.390291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.390434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.390625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.390770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.390970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.391117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.391273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.391414 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.391555 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.391695 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.391836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.392020 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.392178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.392345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.392493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.392637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.392829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.392981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.393156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.393307 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.393449 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.393597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.393748 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.393898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.394231 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.394558 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.395310 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.395482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.395646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.395790 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.395933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.396188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.396385 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.398311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.398573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.398730 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.398888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.399089 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.399739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.399883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.400034 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.400177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.400352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.400540 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.400687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.400880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.401951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.402110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.402269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.402269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.402647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.402853 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.403031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.403176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.403318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.403458 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.404119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.404284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.404481 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.404628 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.404834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.405783 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.405966 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.406160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.406313 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.406528 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.406710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.406874 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.407227 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.407383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.407527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.407680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.407821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.407961 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.408112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.408296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.408476 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.408625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.408804 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.408958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.409108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.409257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.409399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.409549 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.409690 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.409838 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.409993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.410142 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.410328 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.410499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.410658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.410807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.410961 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.411103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.411254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.411402 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.411543 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.411765 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.411910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.412737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.412962 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.413109 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.413259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.413409 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.413552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.413700 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.413843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.413991 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.413991 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.414268 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.414410 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.414650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.414802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.414958 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.414958 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.416082 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.416237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.416380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.416569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.416745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.416899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.417102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.417401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.417563 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.417706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.417861 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.418023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.418173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.418325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.418510 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.418670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.418819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.418969 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.419564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.419721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.419863 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.420013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.420155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.420296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.420468 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.420655 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.420807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.420957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.421140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.421296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.421437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.421578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.421718 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.421865 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.422013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.422164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.422321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.422504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.422713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.422713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.423039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.423197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.423339 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.423480 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.423619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.423760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.423907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.424062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.424211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.424364 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.424505 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.424696 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.424857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.425006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.425155 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.425298 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.426106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.426264 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.426422 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.426610 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.426755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.426896 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.427054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.427204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.427535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.427694 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.427837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 18 00:56:06.428003 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.428149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.428299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.428299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.428584 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.428761 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.428914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.429100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.429261 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.429403 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.429552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.430141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.430295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.430437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.430664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.430819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.430969 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.431110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.431251 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.431400 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.431569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.431713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.431854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.432003 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.432144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.432298 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.432440 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.432579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.432579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.432896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.433093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.433250 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.433393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.433551 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.433709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.433860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.434008 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.434159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.434299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.434449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.434667 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.435102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.435359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.435635 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.436452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.436609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.437946 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.438119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.438280 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.438280 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.438593 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.438744 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.438887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.439029 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.439183 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.439324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.439471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.439613 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.439765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.439913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.440066 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.440207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.440355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.440496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.440496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.440771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.440959 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.441105 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.441105 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.442835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.444424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.444588 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.444738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.444880 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.445037 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.445188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.445360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.445509 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.445768 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.446630 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.446984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.447283 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.447593 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.447912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.448146 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.448300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.448300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.448587 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.448737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.448886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.449081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.449235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.449383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.449534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.449682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.449822 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.449975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.450115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.450115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.450115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.450524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.450681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.450832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.450991 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.451212 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.451364 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.451508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.451658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.451808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.451949 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452096 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452387 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452817 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452976 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.452976 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.453297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.453453 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.453596 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.453737 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.453737 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.454019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.454161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.454161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.454447 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.454661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.454922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.455076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.455076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.455359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.455514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.455514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.455808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.457688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.457688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.457688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.457688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.457688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.457688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.458825 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.458983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.459140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.459359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.460055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.460217 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.460366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.460514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.460661 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.460809 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.460966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.461114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.461268 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.461486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.461638 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.461794 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.461951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.462101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.462267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.462424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.462424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.462763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.462936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.462936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.463229 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.463427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.463427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.463723 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.463880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.464034 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.464201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.464201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.464491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.464647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.464647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.464936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.465121 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.465274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.465421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.465421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.465714 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.465867 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.465867 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.466279 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.466942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.467119 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.467278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.467278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.467561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.467716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.467716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.468012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.468337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.468492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.468640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.468640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.468951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.469108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.469108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.469406 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.469554 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.469701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.469857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.469857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.470148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.470296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.470444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.470671 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.470867 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.471026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.471174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.471321 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.471477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.471638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.471638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.471926 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.472073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.472219 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.472366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.472366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.472366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.472793 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.472953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.473115 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.473263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.473462 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.473462 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.473765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.473765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.474052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.474200 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.474394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.474578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.474735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.474943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.475097 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.475259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.475414 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.475570 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.475720 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.475877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.476026 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.476186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.476429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.477061 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.477232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.477382 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.477542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.477707 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.477856 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.478002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.478160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.478484 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.478642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.478839 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.478990 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.479146 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.479302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.479451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.479606 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.479782 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.479933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.480081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.480235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.480390 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.480551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.480706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.480856 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481004 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.482885 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.483121 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.483274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.483429 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.483622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.483773 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.483921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.484085 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.484247 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.484415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.484571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.484719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.484875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.485030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.485178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.485332 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.485332 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.485657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.485824 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.485981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.486138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.486293 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.486581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.487304 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.488074 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.488239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.488419 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.488747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.488897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.489045 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.489201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.489351 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.489513 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.489691 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.489840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.489989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.490138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.490287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.490485 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.490643 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.490810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.490960 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.491108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.491108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.491108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.491621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.491782 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.491930 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.492078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.492253 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.492465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.492625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.492773 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.492921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.493075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.493233 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.493390 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.493540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.493749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.493915 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.494066 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.494227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.494375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.494597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.494761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.494954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.495117 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.495266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.495413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.495570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.495733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.495907 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.496071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.496236 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.496391 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.496567 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.497128 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.497815 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.497815 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.498127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.498275 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.498423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.498610 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.498986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.499164 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.499321 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.499470 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.499625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.499777 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.499938 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.500093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.500242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.500622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.500793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.500945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.501101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.501250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.501418 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.501568 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.501733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.502622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.502731 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.502819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.502974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.502974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503317 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503736 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503826 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503909 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.503987 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504073 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504320 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504400 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504480 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504656 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504736 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504824 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504824 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.504981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505250 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.505922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.506701 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.506801 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.506961 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.507138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.507226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.507552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.507718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.507807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.507889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.507971 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.508058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.508153 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.508237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.508319 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.508389 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.510552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.510668 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.510768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.510846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.510927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511007 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511091 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511183 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511344 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511424 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511804 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.511959 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512056 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512215 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512456 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512527 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512762 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512902 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.512978 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513638 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513715 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.513715 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.514515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.514613 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.514687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.514756 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.514833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.514912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.514991 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515062 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515132 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515403 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515476 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515630 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.515931 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516128 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516212 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516212 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516212 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516451 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516859 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.516935 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517029 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517276 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517780 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.517940 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518020 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518092 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518169 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518246 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518317 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518386 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518716 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.518871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.519357 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.519493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.519613 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.519733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.519844 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.519952 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520061 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520289 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520520 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520640 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520764 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.520983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521092 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521200 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521460 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521715 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.521994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.522104 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.522223 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.522334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.522450 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.522450 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.522734 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.522867 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.523006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.523136 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.523265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.523401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.523538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.523681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.523850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.524062 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.524227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.524386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.524546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.524708 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.524862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.525013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.525164 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.525314 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.525473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.525637 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.525820 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.525976 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.526127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.526286 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.526445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.526791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.526944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.527096 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.527096 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.527384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.527550 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.527550 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.527550 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.527986 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.528138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.528289 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.528450 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.528603 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.528754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.528754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.529052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.529206 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.529582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.529744 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.529744 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.530052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.530206 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.530358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531096 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531189 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531708 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531792 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531882 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.531974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532064 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532231 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532419 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532518 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.532987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533208 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533382 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533464 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533767 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.533931 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534367 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534492 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534594 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534871 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.534954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535042 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535388 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535523 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535656 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.535970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.536249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.536347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.536576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.536684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.536902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.537002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.537265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.537890 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.538136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.538262 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.538472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.538571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.538827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.538944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.539147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.539267 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.539473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.539706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.539810 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.539903 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.540107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.540224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.540439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.540562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.540764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.540879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.541085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.541187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.541447 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.541539 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.541777 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.541869 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.542142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.542353 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.542574 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.542726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.542821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.543061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.543193 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.543413 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.543519 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.543752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.543843 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.544067 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.544166 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.544385 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.544478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.544701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.544815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.545036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.545137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.545358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.545459 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.545682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.545774 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.546003 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.546104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.546324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.546416 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.546673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.546774 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.546994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.547099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.547313 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.547405 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.547634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.547725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.547954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.548054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.548273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.548408 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.548592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.548696 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.548908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.549007 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.549223 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.549314 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.549533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.549623 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.549853 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.549949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.550160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.550251 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.550517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.550629 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.550819 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.550918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.551140 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.551243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.551472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.551573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.551791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.551881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.552106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.552196 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.552423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.552522 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.552743 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.552832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.553107 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.553708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.553928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.554034 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.554243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.554335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.554617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.554724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.554932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.555063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.555263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.555388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.555601 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.555721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.555937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.556037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.556256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.556349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.556598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.556692 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.556934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.557033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.557260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.557351 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.557578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.557683 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.557902 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.558000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.558254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.558359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.558574 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.558684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.558890 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.558993 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.559211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.559310 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.559524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.559626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.559844 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.559984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.560156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.560267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.560474 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.560580 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.560783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.560875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.561095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.561188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.561405 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.561497 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.561717 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.561809 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.562031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.562128 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.562341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.562440 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.562680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.562784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.562990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.563081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.563297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.563395 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.563606 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.563919 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.564153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.564255 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.564546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.564670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.564863 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.565180 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.565429 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.565531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.565796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.566040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.566140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.566415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.566695 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.566798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.567072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.567290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.567429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.567713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.567889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.568053 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.568402 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.568536 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.569037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.569375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.569477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.569699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.569958 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.570085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.570341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.570565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.570689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.571026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.571345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.571512 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.571689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.572031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.572131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.572469 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.572584 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.572795 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.573120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.573217 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.573753 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.573909 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.574089 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.574437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.574633 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.574920 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.575022 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.575267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.575593 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.575682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.575918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.576188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.576406 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.576613 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.577043 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.577151 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.577392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.577646 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.577745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.578127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.578230 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.578632 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.578719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.578964 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.579044 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.579339 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.579452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.579688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.579805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.580055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.580184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.580394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.580508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.580725 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.580835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.581051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.581152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.581376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.581476 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.581705 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.581811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.582035 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.582127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.582375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.582509 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.582741 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.582845 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.583100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.583225 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.583437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.583538 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.583776 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.583877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.584134 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.584258 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.584489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.584603 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.584848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.584948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.585186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.585295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.585508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.585616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.585828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.585928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.586141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.586243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.586393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120832) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 00:56:06.586526 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.586810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.587161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.587500 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.587828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.588156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.588491 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.589033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.589383 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.589753 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.590112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.590490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.590848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.591191 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.591537 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.591857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.592180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.592525 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.592877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.593222 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.593559 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.593882 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.594200 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.594565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.594888 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.595434 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.595794 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.596122 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.596461 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.596800 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.597130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.597450 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.597783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.598102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.598427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.598776 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.599099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.599419 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.599735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.600592 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.600950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.601430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.601782 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.602097 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.602437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.602779 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.603098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.603413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.603736 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.604052 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.604368 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.604687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.605000 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.605340 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.605658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.605981 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.606315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.606667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.607030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.607352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.607669 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.608002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.608325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.608640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.609141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.609474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.609821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.610136 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.610482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.610838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.611166 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.611726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.612073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.612769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.613112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.613430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.613747 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.614063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.614402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.614747 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.615070 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.615392 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.616297 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.616618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.616942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.617256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.617577 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.617895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.618211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.618555 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.618873 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.619191 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.619508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.619828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.620184 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.620514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.620839 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.621163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.621488 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.621805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.622123 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.622439 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.622786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.623110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.623447 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.623769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.624081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.624804 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.625137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.625472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.625809 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.626134 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.626498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.626827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.627153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.627486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.627808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.628151 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.628508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.628850 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.629189 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.629507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.629827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.630142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.630475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.630805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.631161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.631481 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.631794 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.632112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.632424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.632739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.633075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.633411 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.633754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.634089 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.634424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.634812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.635192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.635495 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.635862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.636204 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.636537 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.636881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.637214 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.637551 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.637859 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.638226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.638639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.638986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.639348 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.639692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.640030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.640351 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.640686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.641026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.641353 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.641677 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.642020 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.642340 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.642705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.643050 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.643411 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.643755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.644076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.644414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.644751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.645065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.645402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.645726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.646057 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.646379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.646759 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.647086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.647403 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.647723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.648039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.648363 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.648688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.649003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.649323 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.649641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.649970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.650290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.650648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.651069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.651296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.651524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO dbcounter [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Registered counter for database nova_api Sep 18 00:56:06.651643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.651944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.652622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.652975 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.653311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.653632 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.653952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.654264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 18 00:56:06.654535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120831) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 18 00:56:06.672699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_db.sqlalchemy.engines [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120832) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:06.680155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writer thread running {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:06.744689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.context [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e1feb29a-fb47-481b-b99a-721bee8616c1(cell1) {{(pid=120832) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 00:56:06.751605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:56:06.752353 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:56:06.755036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:56:06.756409 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:56:06.757220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:56:06.761201 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:56:06.763976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO dbcounter [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Registered counter for database nova_cell0 Sep 18 00:56:06.768703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO dbcounter [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Registered counter for database nova_cell1 Sep 18 00:56:06.777314 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_db.sqlalchemy.engines [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120832) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:06.779363 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_db.sqlalchemy.engines [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120832) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:06.785063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writer thread running {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:06.790997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writer thread running {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:06.813222 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO dbcounter [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Registered counter for database nova_api Sep 18 00:56:06.833377 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_db.sqlalchemy.engines [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120831) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:06.840792 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writer thread running {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:06.899444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.context [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),e1feb29a-fb47-481b-b99a-721bee8616c1(cell1) {{(pid=120831) load_cells /opt/stack/nova/nova/context.py:464}} Sep 18 00:56:06.903343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:56:06.903828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:56:06.905423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:56:06.906174 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:56:06.906664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:56:06.908675 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:56:06.910722 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO dbcounter [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Registered counter for database nova_cell0 Sep 18 00:56:06.914900 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO dbcounter [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Registered counter for database nova_cell1 Sep 18 00:56:06.922230 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_db.sqlalchemy.engines [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120831) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:06.924337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_db.sqlalchemy.engines [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120831) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:06.932933 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writer thread running {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:06.933277 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writer thread running {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:06.937138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO dbcounter [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Registered counter for database nova_cell0 Sep 18 00:56:06.944732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_db.sqlalchemy.engines [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120832) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:06.949261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writer thread running {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:07.088054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO dbcounter [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Registered counter for database nova_cell0 Sep 18 00:56:07.096977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_db.sqlalchemy.engines [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120831) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 18 00:56:07.103427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writer thread running {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 18 00:56:07.272841 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: WARNING keystonemiddleware.auth_token [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:56:07.280222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: WARNING keystonemiddleware.auth_token [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:56:07.435661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: WARNING keystonemiddleware.auth_token [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:56:07.443511 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: WARNING keystonemiddleware.auth_token [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:56:07.630902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: WARNING keystonemiddleware.auth_token [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:56:07.638152 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: WARNING keystonemiddleware.auth_token [None req-989efd04-0e98-4fe4-a35d-d7d3c14f6b66 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:56:07.643863 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562e774de9c0 pid: 120832 (default app) Sep 18 00:56:07.799561 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: WARNING keystonemiddleware.auth_token [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 18 00:56:07.806274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: WARNING keystonemiddleware.auth_token [None req-998ca3aa-86e8-4f67-a430-5319c6ab3694 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 18 00:56:07.811806 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x562e774de9c0 pid: 120831 (default app) Sep 18 00:56:45.522691 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:45.523907 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:45.524970 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:45.526500 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:47.093053 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-640d699f-f218-4164-ba63-f2631a384ab4 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Action: 'create', calling method: .version_select at 0x7fcab2c2d4c0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-2096982120", "metadata": {"Type": "work"}}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:56:50.040929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-640d699f-f218-4164-ba63-f2631a384ab4 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 4.540465 Sep 18 00:56:50.075647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 1/1] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 18 00:56:45 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 4545 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 18 00:56:50.561895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cdd1e8c7-d882-4984-8f8f-c7d02cbd8d78 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Calling method '.version_select at 0x7fcab2c69ee0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:56:50.680866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:50.682364 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:50.685187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:50.686318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:56:52.446078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cdd1e8c7-d882-4984-8f8f-c7d02cbd8d78 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619" status: 200 len: 354 microversion: 2.1 time: 2.358146 Sep 18 00:56:52.446078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 1/2] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 00:56:50 2022] GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619 => generated 354 bytes in 2365 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 00:56:52.497185 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c13d39b9-39cd-4060-833a-27fcb3beb1e4 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Calling method '.version_select at 0x7fcab2afd3a0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:56:52.621245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c13d39b9-39cd-4060-833a-27fcb3beb1e4 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619" status: 200 len: 354 microversion: 2.1 time: 0.140463 Sep 18 00:56:52.622859 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 00:56:52 2022] GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619 => generated 354 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:56:52.660711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b6744308-0e84-49c5-8daf-418d482427f7 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Calling method '.version_select at 0x7fcab2aa5310>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:56:52.820723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b6744308-0e84-49c5-8daf-418d482427f7 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619" status: 202 len: 0 microversion: 2.1 time: 0.165493 Sep 18 00:56:52.821758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 00:56:52 2022] DELETE /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619 => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 00:56:52.837198 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a1c14f80-483e-4735-ade5-78978e9c9259 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Calling method '.version_select at 0x7fcab2afda60>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:56:52.973054 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a1c14f80-483e-4735-ade5-78978e9c9259 tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619" status: 200 len: 353 microversion: 2.1 time: 0.145312 Sep 18 00:56:52.974072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 3/5] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 00:56:52 2022] GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619 => generated 353 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:56:53.997986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-64bcf907-108a-4178-8c96-e1d4c539223b tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Calling method '.version_select at 0x7fcab2aa53a0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:56:54.062350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-64bcf907-108a-4178-8c96-e1d4c539223b tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] HTTP exception thrown: Volume c283fb23-21f7-4816-a666-dc370977e619 could not be found. Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-64bcf907-108a-4178-8c96-e1d4c539223b tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Request method failure captured: Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619 HTTP/1.1 Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzNaL4jf9hckWaiFirWDyTwWKvACZg-EMWaQNN4Gn2Eagul6DmsoD0Z-tIXflIeC_szFm8iBn3pBJq9GJe_Gw1KnhbQHytbKLtLZWRgeTyj0sqpcUqx-I2bIa3dJyYRxVfeH_Y2JkTmhGBqyd_0xFXyt4LNGzkWAhvRm_eQAlkzBvU Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: ee923b3faeca4113af384cc85257b516 Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesGetTestJSON-874988731 Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 00:56:54.073862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/ee923b3faeca4113af384cc85257b516"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/ee923b3faeca4113af384cc85257b516"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_ee923b3faeca4113af384cc85257b516"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/ee923b3faeca4113af384cc85257b516"}]}] Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesGetTestJSON-874988731 Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: ee923b3faeca4113af384cc85257b516 Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesGetTestJSON-874988731 Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesGetTestJSON-874988731-project Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 30aa97b3fe804a90b252b3046d3ff401 Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesGetTestJSON-874988731-project Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Volume c283fb23-21f7-4816-a666-dc370977e619 could not be found. Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'c283fb23-21f7-4816-a666-dc370977e619'} Sep 18 00:56:54.075648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume c283fb23-21f7-4816-a666-dc370977e619 could not be found. (HTTP 404) (Request-ID: req-df735af1-6a70-4f87-9744-03aadd3d82d0) Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:56:54.076254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 00:56:54.077437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume c283fb23-21f7-4816-a666-dc370977e619 could not be found. Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 18 00:56:54.078982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume c283fb23-21f7-4816-a666-dc370977e619 could not be found. Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume c283fb23-21f7-4816-a666-dc370977e619 could not be found. Sep 18 00:56:54.080058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:56:54.088067 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-64bcf907-108a-4178-8c96-e1d4c539223b tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] Returning 404 to user: Volume c283fb23-21f7-4816-a666-dc370977e619 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:56:54.103385 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-64bcf907-108a-4178-8c96-e1d4c539223b tempest-VolumesGetTestJSON-874988731 tempest-VolumesGetTestJSON-874988731-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619" status: 404 len: 109 microversion: 2.1 time: 0.099448 Sep 18 00:56:54.114100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 3/6] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 00:56:53 2022] GET /compute/v2.1/os-volumes/c283fb23-21f7-4816-a666-dc370977e619 => generated 109 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 00:57:02.430313 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1896416697", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "networks": [{"uuid": "4fc3d215-55ee-49c2-92a4-cbf4d73d8975"}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:02.667210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] validate_networks() for [('4fc3d215-55ee-49c2-92a4-cbf4d73d8975', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 00:57:05.221511 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 00:57:05.222214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 00:57:05.222390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 00:57:05.223483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 00:57:05.223903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 00:57:05.278001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:05.278408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:05.278886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:05.279605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:05.280069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:05.280482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:05.419665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Getting quotas for project b94600e628984a14a6abc8764ea5be31. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:57:05.451305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Getting quotas for user 468d75d910874fa8bb0f4dacb589f64c and project b94600e628984a14a6abc8764ea5be31. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:57:05.521825 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 00:57:05.532592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 00:57:05.722428 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2aaace60-de99-489d-9202-450ab829cb4b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.849792 Sep 18 00:57:05.728522 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 4/7] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 00:57:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 3858 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 00:57:05.745884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a9454402-7c44-406e-82b9-4b7ee77ffa0b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:05.746692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a9454402-7c44-406e-82b9-4b7ee77ffa0b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:06.243535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a9454402-7c44-406e-82b9-4b7ee77ffa0b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:06.283649 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a9454402-7c44-406e-82b9-4b7ee77ffa0b tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1215 microversion: 2.1 time: 0.547559 Sep 18 00:57:06.299911 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:05 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1215 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:06.369774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1933175774", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:06.534572 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] validate_networks() for [('a19f83a0-76a4-4dc1-a372-2847ecc4812d', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 00:57:07.313814 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-459e23b2-7d1c-47e1-bfd7-3b3bb6ccbd32 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:07.315289 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-459e23b2-7d1c-47e1-bfd7-3b3bb6ccbd32 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:07.885163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-459e23b2-7d1c-47e1-bfd7-3b3bb6ccbd32 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:07.904745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-459e23b2-7d1c-47e1-bfd7-3b3bb6ccbd32 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1215 microversion: 2.1 time: 0.595370 Sep 18 00:57:07.905763 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 5/9] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:07 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1215 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:08.929899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:08.930643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:08.949021 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:08.952236 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:08.952902 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:08.953857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:08.981478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.027s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:08.984137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:09.595160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:09.626846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-be6be856-14d6-48d8-a140-019ea4f6b02c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1219 microversion: 2.1 time: 0.701360 Sep 18 00:57:09.632335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 6/10] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:08 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1219 bytes in 708 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:09.918633 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 00:57:09.919147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 00:57:09.919406 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 00:57:09.919881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 00:57:09.920332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 00:57:09.959428 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:09.963750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:09.963750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:09.964079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:09.964471 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:09.964902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:10.066332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for project df01daf86854497fad1d1d016c269ef1. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:57:10.112507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for user 54d4e437036640b58fda503c4d213f5c and project df01daf86854497fad1d1d016c269ef1. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:57:10.209620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 00:57:10.218104 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 00:57:10.419868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9373ab3a-d58e-4d33-ad35-68748a8ada70 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.593890 Sep 18 00:57:10.442117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 5/11] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 00:57:05 2022] POST /compute/v2.1/servers => generated 383 bytes in 4601 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 00:57:10.465227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3861a24f-b8f1-4add-9fd3-c646bb9cf773 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:10.469430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3861a24f-b8f1-4add-9fd3-c646bb9cf773 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:10.892078 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3861a24f-b8f1-4add-9fd3-c646bb9cf773 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:10.936809 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3861a24f-b8f1-4add-9fd3-c646bb9cf773 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1222 microversion: 2.1 time: 0.479163 Sep 18 00:57:10.939638 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:10 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1222 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:10.957081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:10.978399 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:11.003510 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:11.004141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:11.004599 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:11.006963 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:11.007726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:11.010406 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:11.412233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4f03043d-3441-4f9c-abf6-9f73f1823b44 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'create', calling method: .version_select at 0x7fcab29c7f70>, body: {"keypair": {"name": "tempest-keypair-2071684712", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAJt0nz3VUc8Hyv845QI+x+SWZD+FPirlep2AIJwvM5Bfz1Gd1R12xyIOiqLSvCQ1bZWjNgCBTMSqe5CPYW6Ag9v0FNoRbDwJLBQ+6lTpiObBrjMe1FQKaaifhF67yZKgQ=="}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:11.663125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:11.684451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-4f03043d-3441-4f9c-abf6-9f73f1823b44 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for project 505bd70415a04f53a64645eb7959a5fb. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:57:11.719482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a6ae62e4-e1be-4423-b86a-bbba5bb0d834 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1273 microversion: 2.1 time: 0.772799 Sep 18 00:57:11.731542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-4f03043d-3441-4f9c-abf6-9f73f1823b44 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for user 294ae98a8492468893beb3b8c2870d5e and project 505bd70415a04f53a64645eb7959a5fb. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:57:11.757459 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:10 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1273 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:11.794591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2006706204", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:11.983451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4f03043d-3441-4f9c-abf6-9f73f1823b44 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 1.572660 Sep 18 00:57:11.996394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 7/14] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Sep 18 00:57:10 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 1592 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Sep 18 00:57:12.033369 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] validate_networks() for [('a19f83a0-76a4-4dc1-a372-2847ecc4812d', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 00:57:12.960506 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3ca31500-e686-40d6-9263-c96a6a056fb6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: .version_select at 0x7fcab2903040>, body: {"keypair": {"name": "tempest-keypair-522796901", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJZmX4ZTc/WWYmygwLjNpw1ACvJQa5ojz0KbelX4uJxDrIOK59J5q3jWI9Iz/S+bbe4FOUlkw8Jfg8Jr8jvWuLJqQkWk2nYe34S6T8IE/r8xelUHTmw46Zv0isrlz5LABA=="}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:13.002235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-3ca31500-e686-40d6-9263-c96a6a056fb6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:57:13.012000 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-3ca31500-e686-40d6-9263-c96a6a056fb6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:57:13.062662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3ca31500-e686-40d6-9263-c96a6a056fb6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.452223 Sep 18 00:57:13.066227 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 8/15] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 00:57:12 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:13.072412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:13.073415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:13.084665 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:13.089449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:13.090773 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:13.091707 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:13.092326 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:13.092972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:13.466129 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:13.476790 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a4310782-19ed-4c18-a0dd-9b79176891d6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1273 microversion: 2.1 time: 0.408959 Sep 18 00:57:13.477448 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 9/16] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:13 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1273 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:14.505245 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:14.506522 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:14.518924 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:14.519432 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:14.519889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:14.524135 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:14.525784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:14.526233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:14.770327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 00:57:14.771525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 00:57:14.772232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 00:57:14.772860 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 00:57:14.773486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 00:57:14.816019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:14.830531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:14.835262 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:14.836220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:14.836796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:14.838306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:14.930297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for project df01daf86854497fad1d1d016c269ef1. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:57:14.945332 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:14.958352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for user 54d4e437036640b58fda503c4d213f5c and project df01daf86854497fad1d1d016c269ef1. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:57:14.961292 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f9b4c8c4-51c9-4547-b1c1-0a64097cf8d7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1273 microversion: 2.1 time: 0.460370 Sep 18 00:57:14.962722 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 10/17] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:14 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1273 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:14.999989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 00:57:15.010739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 00:57:15.247748 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c26f4420-7477-401f-94e0-70d47b731608 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.452701 Sep 18 00:57:15.250909 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 8/18] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 00:57:11 2022] POST /compute/v2.1/servers => generated 383 bytes in 3463 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 00:57:15.268633 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ffba2cfb-06c3-484a-a940-0361decc5b43 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:15.270325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ffba2cfb-06c3-484a-a940-0361decc5b43 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:15.877943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ffba2cfb-06c3-484a-a940-0361decc5b43 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:15.890323 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ffba2cfb-06c3-484a-a940-0361decc5b43 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1222 microversion: 2.1 time: 0.629467 Sep 18 00:57:15.893752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 11/19] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:15 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1222 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:15.991713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:15.993767 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:16.047196 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:16.055208 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:16.058250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:16.069110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:16.074014 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:16.074800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:16.928710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:16.929366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:16.952581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:16.956767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:16.960516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:16.965416 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:16.968529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:16.972020 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:17.453160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:17.475568 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-66a39163-7073-47a8-9e8c-aae9dac9baac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1218 microversion: 2.1 time: 0.560379 Sep 18 00:57:17.480968 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 12/20] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:16 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1218 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:17.650668 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:17.659759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-20e6854d-cd55-47aa-bb62-2b63480ab8f7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 1.677639 Sep 18 00:57:17.666933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 9/21] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:15 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 1687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:18.516082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:18.518209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:18.538113 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:18.541280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:18.542156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:18.543111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:18.545452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:18.548239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:18.689417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:18.690898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:18.704281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:18.705298 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:18.706052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:18.706946 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:18.707547 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:18.708114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:18.948681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:18.962851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-02a5eaa6-d761-4156-b5a9-1d95c21cd635 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1274 microversion: 2.1 time: 0.461236 Sep 18 00:57:18.964667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 13/22] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:18 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1274 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:19.612572 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:19.624019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0d36988f-2d2a-4ea3-aa94-953d14a4f0a2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.938592 Sep 18 00:57:19.624689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 10/23] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:18 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 940 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:19.832015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1083160956", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7ab8f07f-d17d-470a-a9f4-d9a359c2132d"}], "security_groups": [{"name": "tempest-securitygroup--628780002"}], "key_name": "tempest-keypair-2071684712", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:19.987599 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:19.989039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:19.996173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:19.997920 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:19.998688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:20.000066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:20.000872 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:20.001839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:20.243345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:20.258605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-355764f4-ec59-4d1b-a29c-981d0e29d705 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1274 microversion: 2.1 time: 0.275266 Sep 18 00:57:20.276650 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 11/24] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:19 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1274 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:20.306520 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] validate_networks() for [('7ab8f07f-d17d-470a-a9f4-d9a359c2132d', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 00:57:20.680848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:20.683515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:20.699125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:20.699848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:20.700383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:20.701193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:20.702279 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:20.702867 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:22.328654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:22.351227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3842e89f-b847-49ec-bc7f-7d3b7bc4f00f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 1.695300 Sep 18 00:57:22.352831 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 12/25] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:20 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 1707 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:22.379553 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:22.381955 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:22.426286 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:22.427315 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:22.430774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:22.432701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:22.435455 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:22.436095 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:23.417111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:23.439357 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-612a9667-da50-4fe7-b9bd-883389da145e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1280 microversion: 2.1 time: 1.078050 Sep 18 00:57:23.440766 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 13/26] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:22 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1280 bytes in 1084 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:23.952040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-135779bb-90e0-41f0-a59d-37ffe2bf12e0 req-4c627eb9-be76-4a83-9bd4-2bd86d1e9fb3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1886577e-61d1-4954-a392-49f2164992d6", "tag": "1f817878-60ba-4823-9fec-2051e8d7b136"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:23.994849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-135779bb-90e0-41f0-a59d-37ffe2bf12e0 req-4c627eb9-be76-4a83-9bd4-2bd86d1e9fb3 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:23.995331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-135779bb-90e0-41f0-a59d-37ffe2bf12e0 req-4c627eb9-be76-4a83-9bd4-2bd86d1e9fb3 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:24.011242 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-135779bb-90e0-41f0-a59d-37ffe2bf12e0 req-4c627eb9-be76-4a83-9bd4-2bd86d1e9fb3 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:24.088992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-135779bb-90e0-41f0-a59d-37ffe2bf12e0 req-4c627eb9-be76-4a83-9bd4-2bd86d1e9fb3 service nova] Creating event network-changed:1f817878-60ba-4823-9fec-2051e8d7b136 for instance 1886577e-61d1-4954-a392-49f2164992d6 on hv-cinder-maste Sep 18 00:57:24.093038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-135779bb-90e0-41f0-a59d-37ffe2bf12e0 req-4c627eb9-be76-4a83-9bd4-2bd86d1e9fb3 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120832) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 18 00:57:24.093443 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-135779bb-90e0-41f0-a59d-37ffe2bf12e0 req-4c627eb9-be76-4a83-9bd4-2bd86d1e9fb3 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120832) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 18 00:57:24.094639 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:24.097063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:24.099205 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:24.100231 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:24.107226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:24.107933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:24.141147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.compute.rpcapi [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 18 00:57:24.142564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] Releasing lock "compute-rpcapi-router" {{(pid=120832) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 18 00:57:24.164472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 00:57:24.165724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 00:57:24.166473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 00:57:24.167387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 00:57:24.168051 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 00:57:24.180310 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-13ac7f6b-8da0-42c3-951a-6ae999dff3e3 None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.737909 Sep 18 00:57:24.181308 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 14/27] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:57:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:24.185357 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:24.185862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:24.186306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:24.191466 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:24.193840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:24.194886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:24.195530 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:24.195645 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:24.212492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:24.213500 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:24.241493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:24.242661 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:24.244654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:24.251506 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:24.337291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for project 505bd70415a04f53a64645eb7959a5fb. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:57:24.361470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for user 294ae98a8492468893beb3b8c2870d5e and project 505bd70415a04f53a64645eb7959a5fb. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:57:24.424110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 00:57:24.443252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 00:57:24.758422 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-417bc030-cab5-45d6-b7a9-e03b519b1f17 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 4.931812 Sep 18 00:57:24.759458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 14/28] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Sep 18 00:57:19 2022] POST /compute/v2.1/servers => generated 401 bytes in 4933 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 18 00:57:24.765229 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-920715051", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3227bd19-4c4c-45c0-aee5-e73b3870bae1"}], "security_groups": [{"name": "tempest-securitygroup--851872864"}], "key_name": "tempest-keypair-522796901", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:24.962816 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:24.971613 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e8a55b51-6804-42ee-b6ce-85d05ac85ad9 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.784414 Sep 18 00:57:24.978976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 15/29] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:24 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:24.985547 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:25.000137 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:25.007005 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=20 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:25.024513 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:25.026425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:25.037490 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:25.046982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:25.047558 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:25.048080 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:25.696997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:25.705473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-56407e45-c004-46aa-bbe7-f4ecf12a996a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1280 microversion: 2.1 time: 0.723692 Sep 18 00:57:25.710296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 16/30] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:24 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1280 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:25.722947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-35974a57-b2d3-4567-a5af-818bd9c1cfc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:25.725597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-35974a57-b2d3-4567-a5af-818bd9c1cfc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:25.819191 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] validate_networks() for [('3227bd19-4c4c-45c0-aee5-e73b3870bae1', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 00:57:25.903614 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-35974a57-b2d3-4567-a5af-818bd9c1cfc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:25.915190 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-35974a57-b2d3-4567-a5af-818bd9c1cfc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:25.919738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-35974a57-b2d3-4567-a5af-818bd9c1cfc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:25.921025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-35974a57-b2d3-4567-a5af-818bd9c1cfc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1282 microversion: 2.20 time: 0.206432 Sep 18 00:57:25.925281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 17/31] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:25 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1282 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:26.004147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:26.005640 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:26.014278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:26.023616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:26.024467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:26.025455 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:26.026414 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:26.027096 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:26.888319 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:26.924704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-26372ece-320f-4f61-8fe5-d50d1c90f5f1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.920867 Sep 18 00:57:26.942484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 18/32] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:25 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 930 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:26.956143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:26.957002 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:27.008780 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:27.009579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:27.010148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:27.010987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:27.014918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:27.015461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:27.554493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:27.557662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 00:57:27.558161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 00:57:27.558593 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 00:57:27.559717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 00:57:27.560352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 00:57:27.569782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c4629ce4-ba8b-48ae-99f6-0393e4e61e00 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.637971 Sep 18 00:57:27.571852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 19/33] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:26 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:27.572302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:27.573102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:27.573806 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:27.574729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:27.575331 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:27.575900 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:27.596501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:57:27.603407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:57:27.617667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 00:57:27.633062 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 00:57:27.807181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-27d87951-8529-442a-b92d-3d6125f32265 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 3.022982 Sep 18 00:57:27.807181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 15/34] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 00:57:24 2022] POST /compute/v2.1/servers => generated 401 bytes in 3029 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 00:57:27.807181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:27.814909 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:27.858316 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:27.858973 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:27.859433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:27.860287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:27.860657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:27.861070 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:28.042388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-83fc7024-0113-4775-8b03-79a6fb97b7cc req-32a69405-7b73-474c-80f8-886de4190a94 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae00028e-df3f-4c20-84a1-e0e14f4cc457", "tag": "ba2d5bc2-7b64-4dff-96fe-a30ba8d32b9f"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:28.070763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-83fc7024-0113-4775-8b03-79a6fb97b7cc req-32a69405-7b73-474c-80f8-886de4190a94 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:28.070763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-83fc7024-0113-4775-8b03-79a6fb97b7cc req-32a69405-7b73-474c-80f8-886de4190a94 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:28.079421 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-83fc7024-0113-4775-8b03-79a6fb97b7cc req-32a69405-7b73-474c-80f8-886de4190a94 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:28.105616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-83fc7024-0113-4775-8b03-79a6fb97b7cc req-32a69405-7b73-474c-80f8-886de4190a94 service nova] Creating event network-changed:ba2d5bc2-7b64-4dff-96fe-a30ba8d32b9f for instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 on hv-cinder-maste Sep 18 00:57:28.128563 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-83fc7024-0113-4775-8b03-79a6fb97b7cc req-32a69405-7b73-474c-80f8-886de4190a94 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.554631 Sep 18 00:57:28.135581 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 20/35] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:57:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:28.158442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9a5ba238-ec9f-47e2-8f2a-a31b45614472 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:28.159036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9a5ba238-ec9f-47e2-8f2a-a31b45614472 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:28.200085 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:28.218507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:28.223703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:28.228777 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9576178a-1619-4140-91f5-777d1bddeede tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1334 microversion: 2.20 time: 0.438882 Sep 18 00:57:28.249235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 16/36] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:27 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1334 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:28.262781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:28.264778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:28.281024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:28.281534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:28.281976 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:28.284410 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:28.286076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:28.288218 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:28.348063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9a5ba238-ec9f-47e2-8f2a-a31b45614472 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:28.366362 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9a5ba238-ec9f-47e2-8f2a-a31b45614472 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1240 microversion: 2.1 time: 0.218087 Sep 18 00:57:28.394739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 21/37] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:28 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1240 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:28.579780 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:28.591359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-bd6bea89-1bd3-4a5a-a6f7-1805c7f7a38d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.335883 Sep 18 00:57:28.596381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 17/38] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:28 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:28.603763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:28.604768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:28.624505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:28.625342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:28.628674 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:28.630120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:28.633905 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:28.634555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:28.903309 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:28.912945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-71b77dc9-dfe5-436e-80a2-93689a965a5c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.316296 Sep 18 00:57:28.914008 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 22/39] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:28 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:29.263028 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:29.269295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:29.281415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.281906 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.282353 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:29.288111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.288638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.289062 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:29.404936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:29.406684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:29.422032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.423224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.425892 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:29.432000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.434953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.435713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:29.542444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:29.561069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:29.566770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:29.571477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-80681329-1b2d-4fc4-958f-4bb5ed6ceb96 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1334 microversion: 2.20 time: 0.311881 Sep 18 00:57:29.574315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 18/40] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:29 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1334 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:29.612116 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:29.619837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:29.620455 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:29.629872 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.631581 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a9595df0-e5f6-447b-9c05-4a2745d93a7b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1244 microversion: 2.1 time: 0.227422 Sep 18 00:57:29.632838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.634123 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:29.634483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 23/41] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:29 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1244 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:29.635899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.637252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.639259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:29.950122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:29.951216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:29.977985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.978738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.981592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:29.983149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:29.986879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:29.987372 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:30.108836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:30.119306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f9702335-430a-4702-99df-ba9bb1ba81ea tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.504581 Sep 18 00:57:30.123354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 19/42] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:29 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:30.502347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:30.510285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-78965034-ad94-40a1-ac2f-e3c820669381 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.576893 Sep 18 00:57:30.530038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 24/43] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:29 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:30.603677 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:30.606079 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:30.628012 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:30.628662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:30.639475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:30.640620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:30.642698 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:30.643977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:30.665105 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:30.665871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:30.678568 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:30.684311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:30.687129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:30.689386 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:30.699502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:30.699502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:30.897599 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:30.915332 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:30.919404 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:30.922380 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b80ce60d-8909-4a26-86de-fd034f6ba3b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1334 microversion: 2.20 time: 0.321411 Sep 18 00:57:30.923633 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 20/44] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:30 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1334 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:30.956812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:30.968823 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ac4ea4d2-db8d-41e3-9483-95064376e8a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1244 microversion: 2.1 time: 0.310125 Sep 18 00:57:30.969590 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 25/45] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:30 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1244 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:31.145893 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-c3bbb60c-baf5-4496-a55a-5644ceb41209 req-67ac92ce-fc2a-4534-a4e0-190738116eb8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "46b96d5f-1930-4b82-afcb-62726307e170", "tag": "d0ec5e41-6756-47f2-a199-19f6930aa09a"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:31.148018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:31.148732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:31.156645 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.158061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.158803 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.160928 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.161837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.162451 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.174071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c3bbb60c-baf5-4496-a55a-5644ceb41209 req-67ac92ce-fc2a-4534-a4e0-190738116eb8 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.174820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c3bbb60c-baf5-4496-a55a-5644ceb41209 req-67ac92ce-fc2a-4534-a4e0-190738116eb8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.175397 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c3bbb60c-baf5-4496-a55a-5644ceb41209 req-67ac92ce-fc2a-4534-a4e0-190738116eb8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.188472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:31.203597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-c3bbb60c-baf5-4496-a55a-5644ceb41209 req-67ac92ce-fc2a-4534-a4e0-190738116eb8 service nova] Creating event network-changed:d0ec5e41-6756-47f2-a199-19f6930aa09a for instance 46b96d5f-1930-4b82-afcb-62726307e170 on hv-cinder-maste Sep 18 00:57:31.205001 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c3bbb60c-baf5-4496-a55a-5644ceb41209 req-67ac92ce-fc2a-4534-a4e0-190738116eb8 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120831) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 18 00:57:31.205571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c3bbb60c-baf5-4496-a55a-5644ceb41209 req-67ac92ce-fc2a-4534-a4e0-190738116eb8 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120831) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 18 00:57:31.207229 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.207864 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.208485 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.209955 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.210921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.211343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.221413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.compute.rpcapi [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 18 00:57:31.223027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] Releasing lock "compute-rpcapi-router" {{(pid=120831) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 18 00:57:31.242101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b8dcde06-822a-4f81-bd77-429418a2716f None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100953 Sep 18 00:57:31.243588 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 21/46] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:57:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:31.346901 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:31.359342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-597b8fc3-9a4a-4f3f-9ce1-a79e963ff84f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.216034 Sep 18 00:57:31.360575 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 26/47] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:31 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:31.556781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:31.557950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:31.565458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.566153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.566782 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.567781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.568931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.569533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.742150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:31.750096 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c07d2978-5f72-4618-9e83-4f12c23685d3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.198452 Sep 18 00:57:31.751037 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 22/48] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:31 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:31.948158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:31.948875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:31.950760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=9 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:31.958339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.959129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.959699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.960569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:31.961027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:31.961437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:31.999658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:32.000526 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:32.014029 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:32.015095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:32.016517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:32.017922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:32.018746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:32.019693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:32.033231 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:32.043642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:32.044854 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:32.047801 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-27cb5233-ee07-4d34-abed-92318fd99f37 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1334 microversion: 2.20 time: 0.103593 Sep 18 00:57:32.054620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 27/49] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:31 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1334 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:32.107698 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:32.122125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5ca285e3-e88d-4b1c-a80a-586994fd2516 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1244 microversion: 2.1 time: 0.132354 Sep 18 00:57:32.123128 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 23/50] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:31 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1244 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:32.386146 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:32.387270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:32.393871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:32.397170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:32.397821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:32.398922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:32.399542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:32.400117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:32.564241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:32.573154 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-44797807-c1b4-44b1-afe3-897462217189 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.191298 Sep 18 00:57:32.574042 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 28/51] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:32 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:32.773942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:32.774646 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:32.777648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:32.781700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:32.782192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:32.782683 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:32.783383 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:32.783841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:32.784266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:32.970500 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:32.978164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7d1e5f2e-6d5d-4d94-bd88-b15369960ce3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.208807 Sep 18 00:57:32.982492 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 24/52] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:32 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:33.076106 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:33.076828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:33.083537 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:33.084021 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:33.084492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:33.085444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:33.085896 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:33.086312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:33.149935 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:33.150674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:33.152160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:33.157002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:33.157824 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:33.158576 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:33.159512 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:33.160025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:33.160615 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:33.169564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:33.171627 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:33.173158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-dc336358-de5f-4881-af48-a7034587f850 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1334 microversion: 2.20 time: 0.101916 Sep 18 00:57:33.173894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 29/53] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:33 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1334 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:33.234714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:33.243591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-29dc5832-f507-4aa4-8e29-1b29ca7a9d46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1244 microversion: 2.1 time: 0.097620 Sep 18 00:57:33.244618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 25/54] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:33 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1244 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:33.613921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:33.616040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:33.633444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:33.633941 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:33.634370 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:33.635112 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:33.635559 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:33.635977 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:33.886041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:33.899996 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0bb3aef8-9ca9-4285-b0ab-add048c702db tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.301955 Sep 18 00:57:33.900984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 30/55] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:33 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:34.002837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:34.004024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:34.016697 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.017196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.017769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:34.018354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.018966 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.019402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:34.218768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:34.220826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:34.230119 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=8 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:34.233305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.235611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.236821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:34.238146 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.238781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.239792 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:34.243818 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:34.253986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-552a82e7-ef1d-4b3b-989e-fb3c6eeb79ea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.255847 Sep 18 00:57:34.258820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 26/56] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:33 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:34.282233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:34.283167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:34.312399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.313284 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.313939 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:34.314907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.317381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.318753 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:34.340751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:34.372757 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:34.423815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:34.431573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:34.440504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-bf1530f2-f90a-4815-8203-365df58537bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1244 microversion: 2.1 time: 0.164628 Sep 18 00:57:34.440846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:34.446777 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 27/57] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:34 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1244 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:34.451441 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8818fc9a-58d7-4959-911b-507c5436bb52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1334 microversion: 2.20 time: 0.248241 Sep 18 00:57:34.463337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 31/58] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:34 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1334 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:34.930086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:34.931040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:34.944406 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.944914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.945358 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:34.950534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:34.951030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:34.953317 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:35.131599 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:35.139315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-17733f18-55dd-4bf1-bd1f-a3c1df687beb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.215706 Sep 18 00:57:35.139895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 28/59] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:34 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:35.283184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:35.283909 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:35.290813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:35.291295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:35.291723 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:35.292414 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:35.292862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:35.293284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:35.421573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:35.429291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-20c779d4-6ae7-4fb6-a9b3-ae71e5b58857 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.150422 Sep 18 00:57:35.429846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 32/60] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:35 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:35.469948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:35.470498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:35.477251 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:35.477731 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:35.478162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:35.478990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:35.479433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:35.479854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:35.485160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:35.485729 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:35.491630 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:35.492312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:35.492752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:35.493455 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:35.493916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:35.494342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:35.567791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:35.576921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8f7ba69a-4565-4e19-af7a-d3d3496c9afe tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1236 microversion: 2.1 time: 0.110825 Sep 18 00:57:35.577587 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 29/61] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:35 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1236 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:35.588888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:35.597598 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:35.599145 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:35.600237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0befaf47-c5a4-429a-9f21-b661515d99cd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1342 microversion: 2.20 time: 0.118972 Sep 18 00:57:35.600778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 33/62] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:35 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1342 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:57:36.165987 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:36.167159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:36.178559 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.179430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.180244 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.181476 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.182287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.183068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.335572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:36.349847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-32182408-8cf0-45cd-9405-f20a174ea90d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.190660 Sep 18 00:57:36.350928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 30/63] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:36 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:36.452962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:36.453574 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:36.459739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.460236 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.460668 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.461359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.461804 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.462219 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.598666 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:36.602106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:36.602758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:36.607507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9b9d076a-8b7d-4a6e-80fa-fa9ac2605a97 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.159088 Sep 18 00:57:36.608250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 34/64] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:36 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:36.608881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.609585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.610027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.610767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.611223 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.611649 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.628084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:36.629009 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:36.635886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.636424 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.636818 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.637556 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:36.638012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:36.638438 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:36.704866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:36.713760 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f5c32c69-07e4-4760-8f96-514db51d996f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1236 microversion: 2.1 time: 0.116087 Sep 18 00:57:36.714336 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 31/65] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:36 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1236 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:36.736691 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:36.745561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:36.747049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:36.748173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-59fe6014-f6c2-4ad2-a712-d0f3cf02b92a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1342 microversion: 2.20 time: 0.125356 Sep 18 00:57:36.748738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 35/66] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:36 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1342 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:57:37.387297 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:37.387928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:37.393816 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.394293 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.394753 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.395438 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.395883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.396916 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.536071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:37.543990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-742d28aa-ed21-4576-a63e-3d057878f694 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.160486 Sep 18 00:57:37.544571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 32/67] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:37 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:37.634086 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:37.634730 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:37.640783 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.641199 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.641647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.642336 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.642809 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.643224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.739151 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:37.740330 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:37.742360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=26 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:37.747875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.748366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.748811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.749538 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.749974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.750399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.784302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:37.793011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e6f37151-5df8-4834-80f5-81eff17a77c8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.163137 Sep 18 00:57:37.793772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 36/68] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:37 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:37.799030 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:37.799643 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:37.807317 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.807807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.808245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.809193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:37.810063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:37.810865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:37.820944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:37.829215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e314bd1b-f4c9-4e6b-bc4b-9da12f782a0b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1236 microversion: 2.1 time: 0.094388 Sep 18 00:57:37.829791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 33/69] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:37 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1236 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:37.882189 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:37.892462 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:37.893862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:37.895039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f91a0465-9f44-40e2-b160-561e28e79f8b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1342 microversion: 2.20 time: 0.100179 Sep 18 00:57:37.895594 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 37/70] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:37 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1342 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:38.567221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:38.568319 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:38.574746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.575290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.575675 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:38.576420 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.576843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.577347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:38.724056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:38.732483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d6f87d90-10f9-427d-b8a5-1bb9302fc320 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.168906 Sep 18 00:57:38.732940 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 34/71] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:38 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:38.817578 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:38.820244 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:38.823457 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=4 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:38.828372 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.829246 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.829827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:38.831106 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.832326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.833306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:38.852149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:38.853875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:38.860120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.861001 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.861514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:38.862277 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.862829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.863457 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:38.947674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:38.955847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b2ac05ac-6bbf-4ced-80c2-e7a2b065f46d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1236 microversion: 2.1 time: 0.107688 Sep 18 00:57:38.967278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 35/72] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:38 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1236 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:38.967278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:38.967278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:38.976599 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.977101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.977514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:38.978209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:38.978692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:38.979110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:39.049181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:39.052852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:39.058760 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:39.060110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:39.060500 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d316c6bf-a42d-48ad-b889-bd464c6e38b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.246980 Sep 18 00:57:39.061072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 38/73] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:38 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:39.061836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a28507e3-b897-450b-b653-90e7483a2af2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1342 microversion: 2.20 time: 0.103818 Sep 18 00:57:39.062987 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 36/74] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:38 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1342 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:39.756588 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:39.757191 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:39.763239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:39.763715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:39.764234 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:39.764992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:39.765437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:39.765852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:39.902525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:39.910514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-46d183ca-eb0a-4ab8-9a4d-e082de33f6c1 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.158351 Sep 18 00:57:39.911114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 39/75] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:39 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:39.979658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:39.980262 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:39.986185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:39.986708 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:39.987143 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:39.987822 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:39.988264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:39.988678 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:40.048957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:40.056993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a6068a0d-acd0-41c9-9358-4b9783cbee94 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1236 microversion: 2.1 time: 0.081280 Sep 18 00:57:40.057544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 37/76] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:39 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1236 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:40.086513 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:40.087127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:40.093288 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:40.093803 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:40.094243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:40.094968 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:40.095459 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:40.095886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:40.103785 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:40.104622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:40.110609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:40.111200 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:40.111524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:40.112211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:40.112662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:40.115281 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:40.161494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:40.170684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:40.172161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:40.173290 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cea77df8-db1f-486b-b861-df68174d58e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1342 microversion: 2.20 time: 0.091075 Sep 18 00:57:40.183958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 40/77] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:40 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1342 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:40.266287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:40.274877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-abc9c50e-2704-42cf-9adc-7c06cf675428 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.175509 Sep 18 00:57:40.275452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 38/78] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:40 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:40.933097 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:40.933729 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:40.939745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:40.940225 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:40.940661 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:40.941346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:40.941807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:40.942232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:41.080877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:41.081507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:41.088130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:41.088612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:41.089049 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:41.089775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:41.090223 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:41.090716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:41.101683 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:41.109665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ac0d39e0-2245-4e62-b868-990b7f1a8e5f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.180332 Sep 18 00:57:41.110247 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 41/79] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:40 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:41.159427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:41.167827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cb9f0b58-7cf1-4403-9538-2799d97e8a11 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1236 microversion: 2.1 time: 0.090795 Sep 18 00:57:41.168694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 39/80] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:41 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1236 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:41.200461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:41.201173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:41.207081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:41.207565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:41.207997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:41.208864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:41.209320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:41.209743 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:41.277797 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:41.286573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:41.288124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:41.289278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-88e1113f-34ca-4560-bacd-a1044089e22d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1342 microversion: 2.20 time: 0.093067 Sep 18 00:57:41.289848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 42/81] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:41 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1342 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:57:41.307591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:41.308199 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:41.310025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=6 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:41.315471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:41.315989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:41.316426 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:41.317121 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:41.317702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:41.318025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:41.465418 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:41.477740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6ce481ab-b4ca-400a-9876-4c22c51ee552 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.173815 Sep 18 00:57:41.482322 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 40/82] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:41 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:42.136064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:42.136709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:42.144310 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.144805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.145217 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.145964 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.146365 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.147301 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.172147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:42.192708 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:42.193907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:42.205679 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.206177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.206693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.207755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.208403 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.208993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.298826 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:42.317384 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-98245ddd-9eae-4783-96a0-29cffc78052a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1292 microversion: 2.1 time: 0.127094 Sep 18 00:57:42.330697 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 41/83] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:42 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1292 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:42.330697 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:42.330697 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:42.338195 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.338907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.339531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.340430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.341182 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.341600 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.364071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:42.375675 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-63618e45-6e50-4932-9c6c-38d2467b55cf tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.244763 Sep 18 00:57:42.376422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 43/84] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:42 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:42.429786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:42.438757 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:42.440240 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:42.441478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4559717c-02ef-4182-a3ba-d59d08161093 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1340 microversion: 2.20 time: 0.121881 Sep 18 00:57:42.442043 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 42/85] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:42 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1340 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:42.506527 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:42.507155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:42.513582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.514079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.514532 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.515463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:42.515910 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:42.516524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:42.655237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:42.662769 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f8d3cc9c-adc0-4e71-8df7-9b90ae3bc780 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.161307 Sep 18 00:57:42.663335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 44/86] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:42 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:43.347252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:43.347861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:43.354355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.354904 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.355338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.356034 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.356486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.356913 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.398886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:43.399597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:43.405631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.406122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.406576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.407265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.407708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.408122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.423921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:43.427973 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:43.433699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b3f6c876-bdac-4b69-aa00-f4054cec154f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1292 microversion: 2.1 time: 0.090228 Sep 18 00:57:43.434430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 43/87] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:43 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1292 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:43.476669 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:43.477168 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:43.484355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.484906 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.485335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.486030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.486494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.486977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.566515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:43.570425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:43.576073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:43.578395 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:43.579074 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-85a8706e-72db-4120-ae9c-e4eb81b11f51 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.184105 Sep 18 00:57:43.579268 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-93e1ea54-6274-4706-bdd4-cffc8d0de7bd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1340 microversion: 2.20 time: 0.107523 Sep 18 00:57:43.579840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 44/88] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:43 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1340 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:57:43.580351 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 45/89] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:43 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:43.685363 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:43.685975 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:43.692386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.692868 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.693310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.694021 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:43.694504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:43.694941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:43.834783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:43.842386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c93ded35-12d6-4628-8a14-e4403323409b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.161013 Sep 18 00:57:43.842993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 45/90] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:43 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:44.458562 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:44.459231 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:44.465102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.465570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.465995 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:44.466698 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.467147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.467561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:44.529971 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:44.537494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-85ee9da3-2a92-489a-b7f8-7851f1c88820 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1292 microversion: 2.1 time: 0.082766 Sep 18 00:57:44.538046 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 46/91] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:44 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1292 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:44.603856 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:44.604622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:44.609475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:44.610077 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:44.610619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.611104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.611530 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:44.612311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.612762 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.613178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:44.616714 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.617201 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.617639 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:44.618338 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.618820 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.619242 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:44.705663 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:44.713684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:44.715158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:44.716244 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ac984395-d7f8-4b0a-8174-acd5de24489f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1340 microversion: 2.20 time: 0.110644 Sep 18 00:57:44.716798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 47/92] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:44 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1340 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:44.773212 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:44.780886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-fd907c19-d316-4b91-ad46-46c7ad670aa0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.181494 Sep 18 00:57:44.781458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 46/93] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:44 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:44.864665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:44.865266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:44.871266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.871742 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.872174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:44.872887 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:44.873337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:44.873850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:45.013238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:45.020695 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7e1a2a75-4bed-4d31-9204-5ce673760d39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.159919 Sep 18 00:57:45.021321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 48/94] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:44 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:45.560574 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:45.561216 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:45.567306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:45.567790 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:45.568220 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:45.569201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:45.569338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:45.569749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:45.633878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:45.641686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-86f80047-e52e-4b9e-9beb-64d60ef36e79 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1292 microversion: 2.1 time: 0.084987 Sep 18 00:57:45.642341 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 47/95] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:45 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1292 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:45.740975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:45.741576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:45.747570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:45.748040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:45.748470 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:45.749138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:45.749583 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:45.750000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:45.803988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:45.804600 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:45.810734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:45.811287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:45.811647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:45.812343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:45.812798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:45.813223 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:45.814006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:45.822638 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:45.824235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:45.825384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9dd15f97-7937-42d7-9a71-4ab51d8e79da tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1340 microversion: 2.20 time: 0.088241 Sep 18 00:57:45.825967 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 49/96] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:45 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1340 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:45.943707 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:45.951662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2836cf86-cdef-48d7-be4a-163811e93cc3 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.151574 Sep 18 00:57:45.952220 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 48/97] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:45 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:46.044007 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:46.044513 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:46.050492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.050989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.051419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:46.052106 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.052552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.052994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:46.181462 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:46.188923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-916536af-2bc9-4301-ad0b-fd72864b3894 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.148844 Sep 18 00:57:46.189485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 50/98] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:46 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:46.665785 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:46.666446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:46.672805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.673266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.673701 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:46.674423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.674919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.675342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:46.743508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:46.751932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9c25f2c1-7a26-4c36-afb9-72378849a1a7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1292 microversion: 2.1 time: 0.090700 Sep 18 00:57:46.752534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 49/99] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:46 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1292 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:46.861270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:46.861885 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:46.868311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.868875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.869309 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:46.869996 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.870442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.870886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:46.931983 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:46.940116 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:46.941707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:46.943344 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-13248ce5-9109-48c0-ae9d-027db5955f38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1340 microversion: 2.20 time: 0.093484 Sep 18 00:57:46.949220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 51/100] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:46 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1340 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:46.975217 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:46.975829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:46.981620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.982101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.982572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:46.983265 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:46.983718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:46.984145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:47.116276 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:47.124110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e6240867-2632-4e51-adc6-f3ff5bce2a92 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.152724 Sep 18 00:57:47.124671 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 50/101] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:46 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:47.214253 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:47.215095 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:47.223811 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:47.224343 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:47.224773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:47.225464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:47.225914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:47.226350 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:47.361650 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:47.369865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-26cf73a9-9ba5-4401-9a4c-04fe436fa8be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.159729 Sep 18 00:57:47.370433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 52/102] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:47 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:47.776360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:47.776965 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:47.782967 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:47.783445 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:47.783881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:47.784556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:47.785005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:47.785426 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:47.846740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:47.854832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3f489e56-41df-4e5d-ab74-1bdd6ebc3bf0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1292 microversion: 2.1 time: 0.082303 Sep 18 00:57:47.855398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 51/103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:47 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1292 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:47.969417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:47.970069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:47.976125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:47.976600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:47.977027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:47.977706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:47.978151 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:47.978596 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:48.041867 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:48.050195 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:48.051782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:48.053467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1d5aa77f-05a3-470e-bf6d-e61bec3a4df7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1340 microversion: 2.20 time: 0.088150 Sep 18 00:57:48.054309 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 53/104] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:47 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1340 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:48.147503 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:48.148030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:48.153960 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:48.154440 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:48.154904 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:48.155587 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:48.156065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:48.156484 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:48.283672 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:48.291367 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5ee1f5fc-93a4-41bc-a31e-fab70213eb7d tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.147881 Sep 18 00:57:48.291947 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 52/105] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:48 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:48.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:48.394391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:48.402506 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:48.403903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:48.404601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:48.405564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:48.406343 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:48.407932 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:48.699752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:48.714646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-68779452-9f74-40d0-aaf4-13c5a25732aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.325179 Sep 18 00:57:48.715256 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 54/106] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:48 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:48.877024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:48.877630 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:48.883794 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:48.884565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:48.885012 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:48.885704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:48.886242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:48.886625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:48.979370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:48.987025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5efcffa6-bd5b-47a2-a3de-fea14f6062f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1300 microversion: 2.1 time: 0.113755 Sep 18 00:57:48.987591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 53/107] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:48 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1300 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:49.092236 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:49.093660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:49.112520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:49.115243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:49.115882 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:49.117617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:49.119384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:49.122297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:49.289676 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:49.303097 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:49.304606 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:49.305770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1d8e1e8e-ba9d-4f55-a6fe-7675ea75e150 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1340 microversion: 2.20 time: 0.231023 Sep 18 00:57:49.306368 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 55/108] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:49 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1340 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:49.324805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:49.324805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:49.327011 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:49.327511 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:49.327950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:49.328649 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:49.329267 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:49.329840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:49.741737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:49.742923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:49.752245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:49.756700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:49.757543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:49.758160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:49.759258 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:49.759942 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:49.760386 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:49.812051 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ec5f1372-14a9-4a46-8f46-ff47a1e617f4 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.498435 Sep 18 00:57:49.829427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 54/109] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:49 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:50.013851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:50.015338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:50.031931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:50.033262 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:50.033874 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:50.034810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:50.036014 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:50.036646 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:50.460219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:50.468334 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1e95f1b8-4a3f-4051-afbb-963a0672517c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1298 microversion: 2.1 time: 0.459880 Sep 18 00:57:50.475834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:50.481692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 55/110] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:50 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1298 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:50.489762 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1edeb6ff-d1e8-4d16-a845-aab390e274db tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.751589 Sep 18 00:57:50.491622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:50.492100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 56/111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:49 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:50.492598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:50.510386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:50.510936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:50.511378 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:50.512098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:50.512559 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:50.512983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:50.854122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:50.854759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:50.861036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:50.861911 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:50.862617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:50.863773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:50.864224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:50.865322 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:50.915510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:50.942085 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:50.946861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:50.949723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-adf23c17-9f43-466f-af89-b99e281cbc3f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.466238 Sep 18 00:57:50.950941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 56/112] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:50 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:51.228165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:51.244016 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9a1e8587-6d27-4637-9ac3-1b06c36647f2 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.394293 Sep 18 00:57:51.252923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 57/113] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:50 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:51.501897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-7d97d87f-ff2c-4e42-be66-9a5672efdf4f req-d2f19b2d-c38d-4bf3-9544-5e92525c5bdb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5efa25bd-04ff-49f6-9fd5-6e8c82bcd152", "tag": "af5eca8e-a823-4591-9077-2a4b9b7fb155"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:51.507156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:51.510047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:51.523762 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:51.524803 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:51.535674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:51.537203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:51.537766 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-7d97d87f-ff2c-4e42-be66-9a5672efdf4f req-d2f19b2d-c38d-4bf3-9544-5e92525c5bdb service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:51.538837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:51.539573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-7d97d87f-ff2c-4e42-be66-9a5672efdf4f req-d2f19b2d-c38d-4bf3-9544-5e92525c5bdb service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:51.540219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:51.540889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-7d97d87f-ff2c-4e42-be66-9a5672efdf4f req-d2f19b2d-c38d-4bf3-9544-5e92525c5bdb service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:51.587707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-7d97d87f-ff2c-4e42-be66-9a5672efdf4f req-d2f19b2d-c38d-4bf3-9544-5e92525c5bdb service nova] Creating event network-changed:af5eca8e-a823-4591-9077-2a4b9b7fb155 for instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 on hv-cinder-maste Sep 18 00:57:51.598264 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-7d97d87f-ff2c-4e42-be66-9a5672efdf4f req-d2f19b2d-c38d-4bf3-9544-5e92525c5bdb service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100878 Sep 18 00:57:51.599779 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 58/114] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:57:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:51.612391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:51.613320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:51.635923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:51.636431 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:51.636878 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:51.643323 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:51.644303 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:51.644807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:51.900983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:51.915925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-fc37648c-88e1-4551-82e3-d75f16206f6c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.422274 Sep 18 00:57:51.928273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 57/115] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:51 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:51.979144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:51.981567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:51.993746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:51.995021 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:51.995709 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:51.996657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:51.997322 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:51.997948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:52.010282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:52.021417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fd2d8796-1d06-4b10-b627-554a782e2b5d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.416083 Sep 18 00:57:52.022155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 59/116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:51 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:52.200152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:52.213589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:52.215820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:52.217691 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-18b8cbbc-fbd9-4858-bbd7-a3653251789d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.244749 Sep 18 00:57:52.218475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 58/117] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:51 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:57:52.286926 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:52.287766 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:52.297214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:52.298440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:52.299566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:52.300372 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:52.300830 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:52.301249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:52.332524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:52.503111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:52.512632 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3e854537-f346-4e7e-8d0d-56bd7f05829e tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.233901 Sep 18 00:57:52.527675 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 60/118] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:52 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:52.947375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:52.948483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:52.963664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:52.965393 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:52.968911 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:52.970830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:52.971899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:52.972928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:53.049523 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:53.053366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:53.077324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:53.080320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:53.081146 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:53.082447 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:53.083407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:53.084250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:53.424516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:53.451682 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-65b8f544-37f6-4ae3-8ee5-50b42da72f1b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.511941 Sep 18 00:57:53.453949 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 59/119] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:52 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:53.461796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:53.463348 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:53.481986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:53.482768 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:53.483936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:53.485894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:53.487166 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:53.488476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:53.488962 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:53.528349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:57:53.535306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-00310b78-7b27-4236-a3e2-6c0283e4e39f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.493733 Sep 18 00:57:53.543259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 61/120] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:53 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:53.558072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:53.567618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:53.584998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:53.586055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:53.586733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:53.588164 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:53.588828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:53.589440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:54.175019 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:54.184993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:54.186733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:54.187902 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3f82ea0f-7eae-46be-a1fc-e63dd759ae57 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.731910 Sep 18 00:57:54.188534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 60/121] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:53 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:54.210198 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:54.220093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d6297d9b-6fff-4cfa-a903-2d74ce6947cb tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.674258 Sep 18 00:57:54.220897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 62/122] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:53 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:54.323349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-33f1f9fc-c8e7-44de-bc4b-9d61cb0abeb5 req-9502dd2a-008c-4e33-b0da-31a2f07923d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1886577e-61d1-4954-a392-49f2164992d6", "name": "network-vif-plugged", "status": "completed", "tag": "1f817878-60ba-4823-9fec-2051e8d7b136"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:54.334199 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-33f1f9fc-c8e7-44de-bc4b-9d61cb0abeb5 req-9502dd2a-008c-4e33-b0da-31a2f07923d2 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:54.334877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-33f1f9fc-c8e7-44de-bc4b-9d61cb0abeb5 req-9502dd2a-008c-4e33-b0da-31a2f07923d2 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:54.335387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-33f1f9fc-c8e7-44de-bc4b-9d61cb0abeb5 req-9502dd2a-008c-4e33-b0da-31a2f07923d2 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:54.348729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-33f1f9fc-c8e7-44de-bc4b-9d61cb0abeb5 req-9502dd2a-008c-4e33-b0da-31a2f07923d2 service nova] Creating event network-vif-plugged:1f817878-60ba-4823-9fec-2051e8d7b136 for instance 1886577e-61d1-4954-a392-49f2164992d6 on hv-cinder-maste Sep 18 00:57:54.354600 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-33f1f9fc-c8e7-44de-bc4b-9d61cb0abeb5 req-9502dd2a-008c-4e33-b0da-31a2f07923d2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.157304 Sep 18 00:57:54.355746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 61/123] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:57:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:54.477519 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:54.478179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:54.490241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:54.490769 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:54.491303 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:54.492145 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:54.494868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:54.495345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:54.567093 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:54.570304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:54.580402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:54.580900 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:54.581436 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:54.582665 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:54.583215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:54.583873 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:55.094327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:55.095350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:55.107893 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-896ad6c2-c60c-4d36-a73c-74d3dc77367e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.547714 Sep 18 00:57:55.109056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 62/124] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:54 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:55.115263 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2388c5c3-5ffd-45e8-a90f-0ea7d84f0247 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.640647 Sep 18 00:57:55.116000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 63/125] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:54 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:55.133531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-ff5d5759-2bf3-4249-827f-97bb3c2fee6a req-3b82254e-41dc-42e4-ac6f-711ea75d401e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0368fcd1-89d7-4e1f-ad08-1425f3872064", "tag": "190e8866-66ef-440c-9bb4-625459014597"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:55.152057 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ff5d5759-2bf3-4249-827f-97bb3c2fee6a req-3b82254e-41dc-42e4-ac6f-711ea75d401e service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:55.152859 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ff5d5759-2bf3-4249-827f-97bb3c2fee6a req-3b82254e-41dc-42e4-ac6f-711ea75d401e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:55.153639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ff5d5759-2bf3-4249-827f-97bb3c2fee6a req-3b82254e-41dc-42e4-ac6f-711ea75d401e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:55.171996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-ff5d5759-2bf3-4249-827f-97bb3c2fee6a req-3b82254e-41dc-42e4-ac6f-711ea75d401e service nova] Creating event network-changed:190e8866-66ef-440c-9bb4-625459014597 for instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 on hv-cinder-maste Sep 18 00:57:55.178886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-ff5d5759-2bf3-4249-827f-97bb3c2fee6a req-3b82254e-41dc-42e4-ac6f-711ea75d401e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061063 Sep 18 00:57:55.179519 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 63/126] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:57:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:55.217850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:55.218514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:55.225433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:55.225919 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:55.226348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:55.227084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:55.227540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:55.227969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:55.245465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:55.246316 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:55.253599 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:55.254213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:55.254763 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:55.255601 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:55.256329 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:55.256832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:55.448033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:55.456629 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:55.457671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:55.459996 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:55.461277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bbbc6aa9-63ed-46da-ad3d-2967dda58e1b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.247533 Sep 18 00:57:55.461959 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 64/127] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:55 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:55.466647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8306fb81-8866-47cc-b868-fcf9dab22724 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.225645 Sep 18 00:57:55.467282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 64/128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:55 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:56.137602 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:56.138204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:56.144261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.144737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.145162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.145846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.146869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.147287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.151567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:56.152248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:56.158400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.158934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.159401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.160513 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.160628 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.161095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.353292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:56.357011 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:56.361615 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9ef697bf-9351-4cf4-a435-265e615f6f32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.227655 Sep 18 00:57:56.362218 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 65/129] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:56 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:56.366444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c963d258-0f6f-4642-a0de-3c28cfb239c7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.218474 Sep 18 00:57:56.367185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 65/130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:56 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:56.489126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:56.490682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:56.492078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:56.492714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:56.497812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.498302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.498768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.499032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.499470 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.499577 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.499924 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.500025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.500413 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.500654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:56.501098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:56.501516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:56.688150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:56.695188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:56.697014 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:56.698452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:56.699592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-76d7fb19-668d-4258-8ae1-675fb36c7bc3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.216488 Sep 18 00:57:56.700153 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 66/131] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:56 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:56.703411 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f1a03903-7465-4424-b93e-ea3dd3f07ac0 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.215775 Sep 18 00:57:56.704048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 66/132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:56 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:57.394007 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:57.395567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:57.395998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:57.398021 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:57.413130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.416679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.418657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.419149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.421419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.422064 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.423145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.424557 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.425347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.426378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.427186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.428254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.896042 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:57.904895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-53a4b34b-0917-4601-b60c-ceb29e7b78d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.519057 Sep 18 00:57:57.905665 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 67/133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:57 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:57.906257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:57.911441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-7c8efdbf-4afb-4ac1-9fd1-b4bc344d3062 req-158075cb-17df-44f1-9981-395354b7a965 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae00028e-df3f-4c20-84a1-e0e14f4cc457", "name": "network-vif-plugged", "status": "completed", "tag": "ba2d5bc2-7b64-4dff-96fe-a30ba8d32b9f"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:57:57.914917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-50399393-b2bf-40f9-91be-23c9ac576eb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.532636 Sep 18 00:57:57.915328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 67/134] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:57 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:57.921393 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-7c8efdbf-4afb-4ac1-9fd1-b4bc344d3062 req-158075cb-17df-44f1-9981-395354b7a965 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.922152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-7c8efdbf-4afb-4ac1-9fd1-b4bc344d3062 req-158075cb-17df-44f1-9981-395354b7a965 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.922769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-7c8efdbf-4afb-4ac1-9fd1-b4bc344d3062 req-158075cb-17df-44f1-9981-395354b7a965 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.924610 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:57.925566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:57.934894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.935840 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.936475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.937251 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.937768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.938394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.939518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-7c8efdbf-4afb-4ac1-9fd1-b4bc344d3062 req-158075cb-17df-44f1-9981-395354b7a965 service nova] Creating event network-vif-plugged:ba2d5bc2-7b64-4dff-96fe-a30ba8d32b9f for instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 on hv-cinder-maste Sep 18 00:57:57.945650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-7c8efdbf-4afb-4ac1-9fd1-b4bc344d3062 req-158075cb-17df-44f1-9981-395354b7a965 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038086 Sep 18 00:57:57.946263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 68/135] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:57:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:57:57.952573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:57.953639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:57.961509 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.962171 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.962735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:57.963775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:57.964451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:57.965120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:58.250958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:58.253942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:58.259472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7a0ff4b3-9f77-4ab2-9190-fba514b81112 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.342908 Sep 18 00:57:58.260142 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 68/136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:57 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:58.271053 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:58.273325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:58.274810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a7b3b5dc-588b-46af-9555-a8ff33e308d4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.326931 Sep 18 00:57:58.275446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 69/137] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:57 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:58.935651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:58.936333 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:58.942477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:58.944057 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:58.944521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:58.945272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:58.945861 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:58.946790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:58.947641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:58.948210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:58.955655 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:58.956511 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:58.957355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:58.958663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:58.959931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:58.960625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:59.208808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:59.217542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:59.220155 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d6cc6549-0084-44ee-971d-077ba67802f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.285370 Sep 18 00:57:59.234111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cdde527b-9f16-44f6-8310-07f154a761b3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.303341 Sep 18 00:57:59.235348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 69/138] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:58 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:57:59.264656 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 70/139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:58 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:57:59.286530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:59.288114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:59.300048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:57:59.301007 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:57:59.305919 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:59.307052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:59.308676 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:59.309992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:59.310905 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:59.312133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:59.312277 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:59.312420 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:59.312893 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:59.313726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:57:59.314074 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:57:59.314518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:57:59.873286 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:59.884637 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:59.886439 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:59.887733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f7e98212-d004-4b53-84a3-aa6dc3cfec73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.592189 Sep 18 00:57:59.897048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 71/140] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:57:59 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:57:59.940493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:57:59.965055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-234635fe-0737-427f-b34e-919e7b0f6841 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1315 microversion: 2.1 time: 0.679546 Sep 18 00:57:59.966898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 70/141] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:57:59 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1315 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:00.264726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:00.266122 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:00.272723 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:00.273910 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:00.274745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:00.275229 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:00.276372 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:00.277234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:00.277826 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:00.278738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:00.282282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:00.283086 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:00.283726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:00.284676 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:00.285182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:00.285644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:00.855069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:00.864332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fe91c054-6db4-44c9-ac3d-89fea6e50554 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.598224 Sep 18 00:58:00.868033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 71/142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:00 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:00.892167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:00.903715 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-eb01eb6f-dc7d-4f3a-addd-a363c1f53c4f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.644373 Sep 18 00:58:00.904990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 72/143] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:00 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:00.922721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:00.923644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:00.932647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:00.933531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:00.934179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:00.935106 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:00.935758 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:00.936378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:00.995680 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:00.996816 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:01.004368 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:01.004988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:01.005819 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:01.007139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:01.007876 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:01.008548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:01.276865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:01.294936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:01.297641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:01.299058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-39fd5c2c-0084-4d14-997f-2d29a57f69d8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.389773 Sep 18 00:58:01.300235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 72/144] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:58:00 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:58:01.332873 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:01.342579 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-336ea4e9-67e3-4e01-9385-ad8e6e4c9ed6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1502 microversion: 2.1 time: 0.352841 Sep 18 00:58:01.343793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 73/145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:00 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1502 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:01.709116 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-0141ca4d-05c6-4dae-b2ae-9d460a5d1c8f req-8c52b080-11b9-4f28-ac71-74e37d462401 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "46b96d5f-1930-4b82-afcb-62726307e170", "name": "network-vif-plugged", "status": "completed", "tag": "d0ec5e41-6756-47f2-a199-19f6930aa09a"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:01.729605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0141ca4d-05c6-4dae-b2ae-9d460a5d1c8f req-8c52b080-11b9-4f28-ac71-74e37d462401 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:01.732419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0141ca4d-05c6-4dae-b2ae-9d460a5d1c8f req-8c52b080-11b9-4f28-ac71-74e37d462401 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:01.732888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0141ca4d-05c6-4dae-b2ae-9d460a5d1c8f req-8c52b080-11b9-4f28-ac71-74e37d462401 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:01.761357 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-0141ca4d-05c6-4dae-b2ae-9d460a5d1c8f req-8c52b080-11b9-4f28-ac71-74e37d462401 service nova] Creating event network-vif-plugged:d0ec5e41-6756-47f2-a199-19f6930aa09a for instance 46b96d5f-1930-4b82-afcb-62726307e170 on hv-cinder-maste Sep 18 00:58:01.773556 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-0141ca4d-05c6-4dae-b2ae-9d460a5d1c8f req-8c52b080-11b9-4f28-ac71-74e37d462401 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068058 Sep 18 00:58:01.774735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 73/146] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:58:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:58:01.897235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:01.897854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:01.909053 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:01.929401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:01.930005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:01.930814 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:01.931396 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:01.931855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:01.932668 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:01.934936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:01.935478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:01.950149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:01.953806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:01.955974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:01.958186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:01.959597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:01.961906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:02.443099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:02.458878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-24f1230a-d12b-4e46-a523-26cc7dedb42b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.566584 Sep 18 00:58:02.469564 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 74/147] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:01 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:02.489783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:02.492786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:02.509569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:02.510078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:02.512232 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:02.513399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:02.514090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:02.514547 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:02.529616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:02.556098 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-83391247-a9d5-43a9-bbec-9ea47751c4bc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.626041 Sep 18 00:58:02.557080 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 74/148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:01 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:02.784349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:02.794963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:02.796960 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:02.798415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f4a5f283-3616-4c95-b206-34f7202116e4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.326398 Sep 18 00:58:02.799179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 75/149] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:58:02 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:58:03.354345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3950a61c-ee7b-42cf-ae31-1a2918e39458"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:03.357727 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:03.359781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=96 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:03.375277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:03.375277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:03.375277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:03.375277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:03.375600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:03.375600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:03.515376 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:03.516862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:03.542437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:03.543024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:03.543463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:03.545318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:03.546996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:03.548748 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:03.766727 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:03.784469 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4508c70c-bd0f-4e7c-9265-5f5d542d2755 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1322 microversion: 2.1 time: 0.282786 Sep 18 00:58:03.785429 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 76/150] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:03 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1322 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:03.805732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:03.806352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:03.824833 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:03.825384 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:03.825904 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:03.826835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:03.827618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:03.828382 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:04.099716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:04.109353 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d1bb59f6-487a-423f-93ca-acc72af9df19 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.321635 Sep 18 00:58:04.110056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 77/151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:03 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:04.116129 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:04.116938 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:04.124027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:04.134555 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:04.135224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:04.136670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:04.137360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:04.137950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:04.310974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:04.321745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:04.324026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:04.325381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e4501be7-9979-46b2-b6a8-0bea3b00b9f9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.214158 Sep 18 00:58:04.326805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 78/152] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:58:04 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:58:04.331459 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-d32175dc-ef86-48e8-9643-441be85a7587 req-5b1f1c0c-c44e-4bb4-a9fd-a24444f15788 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5efa25bd-04ff-49f6-9fd5-6e8c82bcd152", "name": "network-vif-plugged", "status": "completed", "tag": "af5eca8e-a823-4591-9077-2a4b9b7fb155"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:04.351381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-d32175dc-ef86-48e8-9643-441be85a7587 req-5b1f1c0c-c44e-4bb4-a9fd-a24444f15788 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:04.352078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-d32175dc-ef86-48e8-9643-441be85a7587 req-5b1f1c0c-c44e-4bb4-a9fd-a24444f15788 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:04.352827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-d32175dc-ef86-48e8-9643-441be85a7587 req-5b1f1c0c-c44e-4bb4-a9fd-a24444f15788 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:04.361799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:04.371309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-d32175dc-ef86-48e8-9643-441be85a7587 req-5b1f1c0c-c44e-4bb4-a9fd-a24444f15788 service nova] Creating event network-vif-plugged:af5eca8e-a823-4591-9077-2a4b9b7fb155 for instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 on hv-cinder-maste Sep 18 00:58:04.377549 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-d32175dc-ef86-48e8-9643-441be85a7587 req-5b1f1c0c-c44e-4bb4-a9fd-a24444f15788 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049413 Sep 18 00:58:04.378550 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 79/153] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:58:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:58:04.544007 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-695b0393-4f5e-4c72-a84b-d97d343df665 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "POST /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.193193 Sep 18 00:58:04.545681 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 75/154] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 00:58:03 2022] POST /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6/os-volume_attachments => generated 194 bytes in 1196 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 00:58:04.812573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:04.813261 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:04.821415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:04.822547 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:04.823229 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:04.825542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:04.826108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:04.827571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.017097 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:05.050426 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3d37b475-01b1-4892-881f-1b061c98c868 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1516 microversion: 2.1 time: 0.242978 Sep 18 00:58:05.051473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 80/155] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:04 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1516 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:05.078491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:05.081541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:05.089432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.090230 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.091050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.092140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.092985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.093621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.116771 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:05.129187 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lazy-loading 'flavor' on Instance uuid 46b96d5f-1930-4b82-afcb-62726307e170 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 00:58:05.138058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:05.139127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:05.148811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.149304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.149727 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.151353 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.152086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.152944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.343482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-96db056b-0014-4908-a53f-c821cecea6b5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170/action" status: 200 len: 32 microversion: 2.1 time: 0.271017 Sep 18 00:58:05.350497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 76/156] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 00:58:05 2022] POST /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170/action => generated 32 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 00:58:05.357133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:05.358142 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:05.371928 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.372876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.373840 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.374994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.375700 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.376386 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.423875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:05.441865 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-843c7420-039b-4f99-be13-acaa744ff55e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.309037 Sep 18 00:58:05.442496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 81/157] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:05 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:05.452343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:05.453357 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:05.462680 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.463930 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.464580 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.465515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:05.465978 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:05.466403 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:05.982306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:05.993853 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:06.005039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:06.011836 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:06.016322 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c0c5d610-8cf5-4567-9c85-a1bfaa8d5cdf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.569360 Sep 18 00:58:06.016926 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4a6ca743-6ad1-4727-93c2-e5a27d224bc8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.663303 Sep 18 00:58:06.018379 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 77/158] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:58:05 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:58:06.031358 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 82/159] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:05 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:06.486047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:06.486852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:06.496393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:06.496884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:06.497328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:06.498100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:06.498611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:06.499049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:06.780110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:06.795626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8e114889-5bad-4a61-a7fe-56d3ffd89c9f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.330387 Sep 18 00:58:06.796477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 78/160] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:06 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:07.046162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:07.052620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:07.054758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:07.054997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:07.071576 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:07.072705 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:07.073341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:07.073931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:07.076052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:07.076538 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:07.077607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:07.078438 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:07.079249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:07.080069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:07.080828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:07.082036 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:07.344524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:07.353816 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:07.355798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:07.357142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d210d77b-ab08-4acb-b887-3b13947e2704 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1407 microversion: 2.20 time: 0.315423 Sep 18 00:58:07.366162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:07.370363 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 83/161] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:58:07 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1407 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:58:07.376642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0c5fdb23-7975-45bf-881c-16b28418c485 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.328602 Sep 18 00:58:07.377214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 79/162] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:07 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:07.822619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:07.823720 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:07.837138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:07.839473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:07.841924 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:07.843734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:07.845749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:07.847629 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.021832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-c79bb0d1-1748-4bf2-a800-81e43bcdc7ac req-af502a61-de92-4e2a-94a4-c9d3b2e7fd12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0368fcd1-89d7-4e1f-ad08-1425f3872064", "name": "network-vif-plugged", "status": "completed", "tag": "190e8866-66ef-440c-9bb4-625459014597"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:08.037512 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-c79bb0d1-1748-4bf2-a800-81e43bcdc7ac req-af502a61-de92-4e2a-94a4-c9d3b2e7fd12 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:08.047497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-c79bb0d1-1748-4bf2-a800-81e43bcdc7ac req-af502a61-de92-4e2a-94a4-c9d3b2e7fd12 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:08.048339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-c79bb0d1-1748-4bf2-a800-81e43bcdc7ac req-af502a61-de92-4e2a-94a4-c9d3b2e7fd12 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.070574 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-c79bb0d1-1748-4bf2-a800-81e43bcdc7ac req-af502a61-de92-4e2a-94a4-c9d3b2e7fd12 service nova] Creating event network-vif-plugged:190e8866-66ef-440c-9bb4-625459014597 for instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 on hv-cinder-maste Sep 18 00:58:08.080061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-c79bb0d1-1748-4bf2-a800-81e43bcdc7ac req-af502a61-de92-4e2a-94a4-c9d3b2e7fd12 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064747 Sep 18 00:58:08.081862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 80/163] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:58:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:58:08.167317 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:08.179179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-eb29d290-541b-4ef4-875f-620872214fec tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.363478 Sep 18 00:58:08.187363 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 84/164] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:07 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:08.384448 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:08.385142 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:08.393828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:08.393828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:08.393828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.393828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:08.394265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:08.394265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.401731 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:08.402691 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:08.410158 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:08.411083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:08.412013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.413031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:08.413658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:08.414084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.605944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:08.614304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:08.616269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:08.618674 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:08.620663 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a0f7dce5-d45c-472d-aa73-e3be074ebef8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1599 microversion: 2.20 time: 0.240946 Sep 18 00:58:08.621450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 81/165] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:58:08 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1599 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:58:08.627619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-333b8717-cea0-47f3-8317-b1bf007931d9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.230068 Sep 18 00:58:08.628508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 85/166] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:08 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:08.646749 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:08.647615 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:08.653573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:08.654057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:08.654542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.655510 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:08.656201 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:08.656739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:08.919472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8927c6b3-9ebe-4a4e-9923-77aa1f126ad7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.277073 Sep 18 00:58:08.920088 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 82/167] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Sep 18 00:58:08 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-interface => generated 284 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:58:09.213486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:09.214009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:09.223778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:09.224871 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:09.225482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:09.226651 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:09.227651 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:09.228274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:09.582132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:09.600333 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4ff39523-e20c-4269-8ae1-951f268d3331 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.390652 Sep 18 00:58:09.601257 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 86/168] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:09 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:09.653502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:09.654270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:09.660414 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:09.661121 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:09.661597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:09.662285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:09.662771 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:09.663193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:09.943715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:09.952865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0350f32c-efc9-4a13-9c71-33dbac142572 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.305337 Sep 18 00:58:09.953887 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 83/169] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:09 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:10.621536 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-c53651d8-033a-435e-bfd3-0023a6e3e78a req-aa71866d-b81b-487b-bbd8-5aa272b0ed20 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5efa25bd-04ff-49f6-9fd5-6e8c82bcd152", "tag": "af5eca8e-a823-4591-9077-2a4b9b7fb155"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:10.625350 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:10.626507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:10.643028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:10.643028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:10.643028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:10.643028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:10.643628 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:10.643628 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:10.663494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c53651d8-033a-435e-bfd3-0023a6e3e78a req-aa71866d-b81b-487b-bbd8-5aa272b0ed20 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:10.667354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c53651d8-033a-435e-bfd3-0023a6e3e78a req-aa71866d-b81b-487b-bbd8-5aa272b0ed20 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:10.667804 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-c53651d8-033a-435e-bfd3-0023a6e3e78a req-aa71866d-b81b-487b-bbd8-5aa272b0ed20 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:10.711576 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-c53651d8-033a-435e-bfd3-0023a6e3e78a req-aa71866d-b81b-487b-bbd8-5aa272b0ed20 service nova] Creating event network-changed:af5eca8e-a823-4591-9077-2a4b9b7fb155 for instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 on hv-cinder-maste Sep 18 00:58:10.716867 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-c53651d8-033a-435e-bfd3-0023a6e3e78a req-aa71866d-b81b-487b-bbd8-5aa272b0ed20 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.110116 Sep 18 00:58:10.717523 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 87/170] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:58:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:58:10.811348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:10.821167 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-98376810-48e2-4c64-85b2-2931a2fdd364 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.200124 Sep 18 00:58:10.821919 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 84/171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:10 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:10.977863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:10.978607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:10.984660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:10.985148 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:10.985587 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:10.986280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:10.986758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:10.987186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:11.136114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:11.145005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-836a17e8-88db-487c-9912-c00235c6406a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.171100 Sep 18 00:58:11.145590 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 88/172] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:10 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:11.843945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:11.844554 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:11.851125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:11.851622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:11.852025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:11.853781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:11.853781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:11.853781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:12.027614 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:12.035442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0f1bc050-a624-426e-88d2-282bdd97b1c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.195198 Sep 18 00:58:12.036010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 85/173] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:11 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:12.174390 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:12.176316 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:12.186099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:12.186810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:12.187305 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:12.188424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:12.189022 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:12.189732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:12.472765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:12.482880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f3ebe57c-90cc-4668-97ff-8bff926f5722 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.316441 Sep 18 00:58:12.483493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 89/174] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:12 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:13.062089 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:13.063354 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:13.071665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.072152 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.072842 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.073709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.074159 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.074740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.305098 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:13.312697 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-61c33aa0-976a-4c22-af6b-175cd990fb41 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.255116 Sep 18 00:58:13.313851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 86/175] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:13 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:13.510978 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:13.515830 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:13.516015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:13.517761 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:13.531907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.532617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.532780 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.533666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.534674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.534806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.535131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.536044 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.536665 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.536936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.538935 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.539375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.574397 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 00:58:13.733496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-476e9315-13f4-4673-80e2-942ae21665ee tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 204 len: 0 microversion: 2.1 time: 0.234353 Sep 18 00:58:13.735306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 90/176] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 00:58:13 2022] DELETE /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 18 00:58:13.745670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:13.746372 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:13.753171 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.753792 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.754441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.755341 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:13.755847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:13.756279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:13.831102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:13.849693 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c5a2b307-ed99-45fb-9479-3fc887772c7e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.344717 Sep 18 00:58:13.850274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 87/177] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:13 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:13.978164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:13.990116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d6ac8b26-e3f3-43c5-bc11-14df7fef2f1a tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1554 microversion: 2.1 time: 0.250287 Sep 18 00:58:13.990770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 91/178] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:13 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1554 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:14.339776 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:14.340512 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:14.347311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:14.347906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:14.348443 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:14.349300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:14.349855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:14.350383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:14.540130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:14.548954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-da2e2fb9-8274-437b-94a5-65771aef1356 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.213100 Sep 18 00:58:14.549529 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 88/179] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:14 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:14.873687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:14.875289 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:14.883770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:14.884508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:14.885013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:14.886359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:14.886907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:14.888167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:15.016605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:15.017744 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:15.029266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:15.030262 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:15.030885 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:15.032575 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:15.033337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:15.034319 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:15.131846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:15.157845 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3be32076-fe1b-4b53-a316-c9b5046c1b06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.288788 Sep 18 00:58:15.158482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 92/180] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:14 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:15.265236 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:15.280378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1a35225f-25f6-416a-99e7-a804978985b7 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1554 microversion: 2.1 time: 0.269851 Sep 18 00:58:15.298767 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 89/181] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:15 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1554 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:15.573015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:15.573688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:15.587031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:15.587745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:15.591706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:15.592737 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:15.593460 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:15.594065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:15.758852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:15.774763 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-bbf90567-6814-4041-a917-8f65ddf5cbf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1365 microversion: 2.1 time: 0.206843 Sep 18 00:58:15.775702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 93/182] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:15 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1365 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:16.185836 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:16.186609 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:16.194841 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:16.195423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:16.195937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:16.196737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:16.197269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:16.198158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:16.316781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:16.317423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:16.335545 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:16.337832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:16.339542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:16.341771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:16.344387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:16.345013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:16.469122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:16.483930 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9eb5c674-725d-4a27-b5c4-639910b80f32 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.302531 Sep 18 00:58:16.484896 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 90/183] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:16 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:16.571082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:16.581200 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6e70dc9d-af6a-4aa7-a50d-aa474d16152f tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1554 microversion: 2.1 time: 0.276696 Sep 18 00:58:16.581752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 94/184] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:16 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1554 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:16.804629 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:16.805242 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:16.814529 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:16.815252 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:16.815848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:16.816546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:16.817154 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:16.818179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:16.987798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:17.000340 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ee9237d0-781b-47f6-8eef-b4ae8425134a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1553 microversion: 2.1 time: 0.200007 Sep 18 00:58:17.000928 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 91/185] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:16 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1553 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:17.027783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:17.028660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:17.041686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:17.041686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:17.041686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:17.042594 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:17.043291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:17.043924 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:17.068837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:17.146350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-367542c4-2f2b-4f34-91dd-92887f2b2dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.124947 Sep 18 00:58:17.147360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 95/186] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 00:58:17 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-interface => generated 284 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:58:17.513081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:17.513719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:17.533548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:17.533548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:17.533548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:17.533548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:17.533874 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:17.533874 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:17.543918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:17.605641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:17.607300 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:17.620422 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:17.621215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:17.623615 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:17.624859 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:17.626208 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:17.630907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:17.908622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:17.927154 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-649fa685-63e9-4b15-8e96-a201605aec2b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.420621 Sep 18 00:58:17.927799 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 92/187] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:17 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:18.039767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:18.064153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-25e2de29-5893-44bc-98e0-cee8651ab317 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1554 microversion: 2.1 time: 0.463044 Sep 18 00:58:18.085459 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 96/188] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:17 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1554 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:18.422572 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-626bf514-7275-40bc-8fe5-cd4430858d48 req-b265731c-0ce4-4309-bfa1-6eed13a37110 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1886577e-61d1-4954-a392-49f2164992d6", "name": "network-vif-deleted", "tag": "1f817878-60ba-4823-9fec-2051e8d7b136"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:18.434787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-626bf514-7275-40bc-8fe5-cd4430858d48 req-b265731c-0ce4-4309-bfa1-6eed13a37110 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:18.435348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-626bf514-7275-40bc-8fe5-cd4430858d48 req-b265731c-0ce4-4309-bfa1-6eed13a37110 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:18.435956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-626bf514-7275-40bc-8fe5-cd4430858d48 req-b265731c-0ce4-4309-bfa1-6eed13a37110 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:18.451114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-626bf514-7275-40bc-8fe5-cd4430858d48 req-b265731c-0ce4-4309-bfa1-6eed13a37110 service nova] Creating event network-vif-deleted:1f817878-60ba-4823-9fec-2051e8d7b136 for instance 1886577e-61d1-4954-a392-49f2164992d6 on hv-cinder-maste Sep 18 00:58:18.459526 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-626bf514-7275-40bc-8fe5-cd4430858d48 req-b265731c-0ce4-4309-bfa1-6eed13a37110 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044249 Sep 18 00:58:18.460211 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 93/189] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:58:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:58:18.952858 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:18.954241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:18.966420 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:18.968618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:18.970663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:18.974488 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:18.975292 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:18.977309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:19.063680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-26635485-2fb9-4d0b-8c3a-c7f3452f448c req-d10cc974-87f5-4e1f-be3c-5ccf02bf1fc3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0368fcd1-89d7-4e1f-ad08-1425f3872064", "tag": "190e8866-66ef-440c-9bb4-625459014597"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:19.082806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-26635485-2fb9-4d0b-8c3a-c7f3452f448c req-d10cc974-87f5-4e1f-be3c-5ccf02bf1fc3 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:19.084046 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-26635485-2fb9-4d0b-8c3a-c7f3452f448c req-d10cc974-87f5-4e1f-be3c-5ccf02bf1fc3 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:19.084929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-26635485-2fb9-4d0b-8c3a-c7f3452f448c req-d10cc974-87f5-4e1f-be3c-5ccf02bf1fc3 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:19.103284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-26635485-2fb9-4d0b-8c3a-c7f3452f448c req-d10cc974-87f5-4e1f-be3c-5ccf02bf1fc3 service nova] Creating event network-changed:190e8866-66ef-440c-9bb4-625459014597 for instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 on hv-cinder-maste Sep 18 00:58:19.108868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-26635485-2fb9-4d0b-8c3a-c7f3452f448c req-d10cc974-87f5-4e1f-be3c-5ccf02bf1fc3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049573 Sep 18 00:58:19.109983 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 94/190] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:58:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:58:19.116438 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:19.117314 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:19.125305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:19.125897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:19.126399 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:19.127190 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:19.127707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:19.128214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:19.242992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '468d75d910874fa8bb0f4dacb589f64c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b94600e628984a14a6abc8764ea5be31', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:19.256067 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2e12c3c7-bae7-4e2e-9be9-5be831e96379 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 200 len: 1346 microversion: 2.1 time: 0.142276 Sep 18 00:58:19.256781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 95/191] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:19 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 1346 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:19.362752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:19.384165 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9d317112-514f-4680-9e69-e6d65fb8f4f5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.437350 Sep 18 00:58:19.386735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 97/192] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:18 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:20.282982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:20.283595 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:20.299791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:20.301870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:20.302522 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:20.303407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:20.303867 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:20.304293 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:20.321073 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] HTTP exception thrown: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Request method failure captured: Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 HTTP/1.1 Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzdTH1MQsJz1H-DblTJT2_R4ldErmDO1wzbDM-QHmwGUqHxfo89jqlolHnR9Tjei8Zbkk43TUXQJCFJPkI9dR0duwkxSVWsAPr7oRMoIxB99rCtrFsXCYnJ4FLR_KntHsI8C4laZ40eaR1BxmvJ_UGLS_chciM8qkDIydwTBSfo4B4 Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 00:58:20.328624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94600e628984a14a6abc8764ea5be31"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}] Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 468d75d910874fa8bb0f4dacb589f64c Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.330168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '1886577e-61d1-4954-a392-49f2164992d6'} Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:58:20.331103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.332266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.334808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Returning 404 to user: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:58:20.335889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e657f495-fc7c-4784-8c5c-fa121a597a4c tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 404 len: 111 microversion: 2.1 time: 0.056865 Sep 18 00:58:20.337035 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 96/193] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:20 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 00:58:20.426693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:20.428300 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:20.435977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:20.436879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:20.437472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:20.439552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:20.440325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:20.441162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:20.501652 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:20.504188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:20.515122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:20.515618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:20.516053 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:20.516755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:20.517212 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:20.517631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:20.543299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] HTTP exception thrown: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Request method failure captured: Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6/os-volume_attachments HTTP/1.1 Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzdTH1MQsJz1H-DblTJT2_R4ldErmDO1wzbDM-QHmwGUqHxfo89jqlolHnR9Tjei8Zbkk43TUXQJCFJPkI9dR0duwkxSVWsAPr7oRMoIxB99rCtrFsXCYnJ4FLR_KntHsI8C4laZ40eaR1BxmvJ_UGLS_chciM8qkDIydwTBSfo4B4 Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 00:58:20.548334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94600e628984a14a6abc8764ea5be31"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}] Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 468d75d910874fa8bb0f4dacb589f64c Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.549877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'server_id': '1886577e-61d1-4954-a392-49f2164992d6'} Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:58:20.550864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:20.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:20.557215 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Returning 404 to user: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:58:20.558097 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-988c5ee6-43c9-4973-8c35-361e22c9c6b6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.062596 Sep 18 00:58:20.558693 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 97/194] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 00:58:20 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6/os-volume_attachments => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 00:58:20.644841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:20.666480 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-65fdc834-6b4d-4bba-885c-7ac6caf476fb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.253496 Sep 18 00:58:20.670594 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 98/195] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:20 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:21.695403 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:21.696047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:21.703145 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:21.703627 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:21.704062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:21.704786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:21.705239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:21.705660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:21.835474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:21.836098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:21.842577 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:21.843140 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:21.843815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:21.844585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:21.845107 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:21.845998 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:21.848187 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:21.857246 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-abd5a92f-4905-44d9-b24a-089ec7e79a6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.165799 Sep 18 00:58:21.857958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 98/196] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:21 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:21.861473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] HTTP exception thrown: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Request method failure captured: Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: DELETE /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 HTTP/1.1 Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzdTH1MQsJz1H-DblTJT2_R4ldErmDO1wzbDM-QHmwGUqHxfo89jqlolHnR9Tjei8Zbkk43TUXQJCFJPkI9dR0duwkxSVWsAPr7oRMoIxB99rCtrFsXCYnJ4FLR_KntHsI8C4laZ40eaR1BxmvJ_UGLS_chciM8qkDIydwTBSfo4B4 Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 00:58:21.866234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94600e628984a14a6abc8764ea5be31"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}] Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 468d75d910874fa8bb0f4dacb589f64c Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.867699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '1886577e-61d1-4954-a392-49f2164992d6'} Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:58:21.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.869431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.872040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Returning 404 to user: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:58:21.872040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-baee8bc4-bbf9-48c3-9b8a-4d9eb3ff8745 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 404 len: 111 microversion: 2.1 time: 0.036823 Sep 18 00:58:21.872040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 99/197] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 00:58:21 2022] DELETE /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 00:58:21.876285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:21.876911 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] [instance: 1886577e-61d1-4954-a392-49f2164992d6] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:21.883374 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:21.883997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:21.884472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:21.885502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:21.885706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:21.886159 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:21.900362 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] HTTP exception thrown: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Request method failure captured: Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 HTTP/1.1 Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzdTH1MQsJz1H-DblTJT2_R4ldErmDO1wzbDM-QHmwGUqHxfo89jqlolHnR9Tjei8Zbkk43TUXQJCFJPkI9dR0duwkxSVWsAPr7oRMoIxB99rCtrFsXCYnJ4FLR_KntHsI8C4laZ40eaR1BxmvJ_UGLS_chciM8qkDIydwTBSfo4B4 Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 00:58:21.901870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b94600e628984a14a6abc8764ea5be31"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b94600e628984a14a6abc8764ea5be31"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b94600e628984a14a6abc8764ea5be31"}]}] Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: b94600e628984a14a6abc8764ea5be31 Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-DeleteServersTestJSON-1995478722 Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 468d75d910874fa8bb0f4dacb589f64c Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-DeleteServersTestJSON-1995478722-project Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.903192 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '1886577e-61d1-4954-a392-49f2164992d6'} Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:58:21.903950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. Sep 18 00:58:21.905038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:58:21.905956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] Returning 404 to user: Instance 1886577e-61d1-4954-a392-49f2164992d6 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:58:21.905956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-82a710be-9023-430e-aeeb-8063da3ec4c6 tempest-DeleteServersTestJSON-1995478722 tempest-DeleteServersTestJSON-1995478722-project] 10.222.0.42 "GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6" status: 404 len: 111 microversion: 2.1 time: 0.030988 Sep 18 00:58:21.905956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 99/198] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:21 2022] GET /compute/v2.1/servers/1886577e-61d1-4954-a392-49f2164992d6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 00:58:22.881258 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:22.881856 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:22.887799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:22.888283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:22.888713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:22.889405 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:22.889850 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:22.890269 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:23.021908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:23.031707 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ca84e778-16c2-4573-9392-0ffb887660d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.154119 Sep 18 00:58:23.032668 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 100/199] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:22 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:24.055748 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:24.056836 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:24.063743 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:24.064325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:24.065023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:24.065879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:24.066586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:24.066957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:24.432007 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:24.440385 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-183debec-e98e-42e0-9d0e-160487336e27 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.388505 Sep 18 00:58:24.441034 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 100/200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:24 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:25.465294 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:25.465892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:25.472654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:25.473221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:25.473883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:25.474767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:25.475126 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:25.475549 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:25.643042 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:25.655169 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-313d4073-e59f-46a2-b947-adfe4d1e1266 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.193548 Sep 18 00:58:25.655812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 101/201] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:25 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:26.679521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:26.681935 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:26.692232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:26.693523 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:26.694292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:26.695645 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:26.698015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:26.699095 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:26.925689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:26.938126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3f836132-5be2-457f-a536-0370b2927e9c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.260245 Sep 18 00:58:26.938799 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 101/202] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:26 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:27.967980 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:27.968589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:27.979492 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:27.979980 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:27.980411 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:27.981106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:27.981555 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:27.982153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:28.163238 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:28.171710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a195cbd2-f410-4a23-b47e-ba40f486463d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.208483 Sep 18 00:58:28.172271 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 102/203] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:27 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:29.197157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:29.198671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:29.205481 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:29.205954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:29.206717 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:29.207734 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:29.208718 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:29.209341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:29.410949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:29.422321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-00d3b571-10ec-46bc-b161-73fa82d38b15 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.231545 Sep 18 00:58:29.423123 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 102/204] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:29 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:30.448990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:30.449596 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:30.456244 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:30.456726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:30.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:30.458044 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:30.458698 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:30.459177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:30.613288 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:30.622732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6ff466c9-a82d-4584-8750-17a08798665c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.178361 Sep 18 00:58:30.623309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 103/205] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:30 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:31.646574 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:31.648554 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:31.655463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:31.655947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:31.656379 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:31.657072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:31.657516 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:31.658099 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:31.808967 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:31.819325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-14a47963-b95b-4f33-b654-24d24f5c95b0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.177683 Sep 18 00:58:31.820260 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 103/206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:31 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:32.844510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:32.845335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:32.848493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=64,nova_api:UPDATE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:32.852619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:32.853260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:32.853769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:32.855289 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:32.855861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:32.856362 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:32.996838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:33.011507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e69a11e7-6f32-4be4-84ae-96bb358ba360 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.170467 Sep 18 00:58:33.014836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 104/207] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:32 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:34.049066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:34.049704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:34.052589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=58 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:34.058432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:34.058939 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:34.059505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:34.060215 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:34.060684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:34.061104 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:34.218621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:34.233156 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6b16a14d-32b4-44b8-b84f-1508e8323ae2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.187827 Sep 18 00:58:34.233802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 104/208] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:34 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:35.257125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:35.258488 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:35.265529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:35.266318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:35.266948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:35.267815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:35.268463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:35.269192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:35.431053 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:35.442508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ce818fbc-78ed-449f-91a2-9640b3d34a34 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.186904 Sep 18 00:58:35.442508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 105/209] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:35 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:36.469005 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:36.469621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:36.476476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:36.477037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:36.477377 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:36.478072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:36.478552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:36.479030 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:36.620003 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:36.628258 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7c698fff-8b22-4c6f-ad94-a446b4c761d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.163185 Sep 18 00:58:36.628835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 105/210] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:36 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:37.651650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:37.652283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:37.658359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:37.658884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:37.659309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:37.660005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:37.660454 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:37.660868 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:37.812424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:37.821313 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2796d7e2-9762-4100-ae20-7c13ee039fa2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.173553 Sep 18 00:58:37.821895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 106/211] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:37 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:38.850155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:38.850970 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:38.857327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:38.857945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:38.858635 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:38.859483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:38.860062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:38.860682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:39.006324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:39.016102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4428571b-1c3e-4e9e-85b9-00a24335d348 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.171045 Sep 18 00:58:39.016892 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 106/212] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:38 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:58:40.055723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:40.056380 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:40.063461 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:40.063956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:40.064325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:40.065010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:40.065459 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:40.065875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:40.201170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:40.211205 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-704753ff-7149-4285-aa4e-f3a48449b7b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.159740 Sep 18 00:58:40.211851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 107/213] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:40 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:41.235947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:41.236550 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:41.243516 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:41.244040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:41.244421 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:41.245094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:41.245538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:41.245951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:41.372918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:41.381165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ae7620cc-3dfa-4273-b654-613792b75eb6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.149274 Sep 18 00:58:41.381721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 107/214] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:41 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:42.404410 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:42.405019 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:42.410984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:42.411466 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:42.411898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:42.412591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:42.413041 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:42.413463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:42.551515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:42.559360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-02686735-04b7-40d2-911c-7d84cd6ff33d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.158930 Sep 18 00:58:42.559919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 108/215] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:42 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:43.584130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:43.584813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:43.591230 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:43.591708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:43.592135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:43.592821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:43.593267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:43.593690 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:43.746360 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:43.754317 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ca84eafe-e328-41fd-b90c-81ef0774af41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.175231 Sep 18 00:58:43.754924 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 108/216] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:43 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:44.777401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:44.778090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:44.784035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:44.784522 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:44.784958 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:44.785766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:44.786253 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:44.786710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:44.922889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:44.931038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a1a0ea1e-32d9-4c5f-a865-a5f5166fc374 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.157351 Sep 18 00:58:44.931669 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 109/217] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:44 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:45.956521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:45.957227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:45.963801 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:45.964370 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:45.964901 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:45.965673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:45.966214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:45.966725 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:46.130888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:46.140334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-953a5799-40ef-46e5-8c38-fefa334c2d6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.188728 Sep 18 00:58:46.141067 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 109/218] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:45 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:47.164324 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:47.164945 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:47.182917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:47.183614 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:47.184339 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:47.185194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:47.185963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:47.186560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:47.190383 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=81 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:47.325623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:47.336152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-01f7bc02-0602-4702-b997-6273aff65396 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.176602 Sep 18 00:58:47.336818 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 110/219] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:47 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:48.361290 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:48.362334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:48.369498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:48.370071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:48.370647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:48.371420 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:48.371975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:48.372466 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:48.375420 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=87 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:58:48.516955 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:48.525040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-70c238d1-9727-4a04-99aa-c2e1efa22341 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.168025 Sep 18 00:58:48.525807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 110/220] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:48 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:49.548861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:49.550114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:49.558233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:49.563219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:49.563219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:49.563219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:49.563219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:49.568703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:49.790728 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:49.800050 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0d8472bb-ba01-4784-9c1a-03dd4f831c6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1522 microversion: 2.1 time: 0.255533 Sep 18 00:58:49.800623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 111/221] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:49 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1522 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:50.826299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:50.827420 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:50.834009 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:50.834644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:50.835839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:50.836894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:50.837428 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:50.837925 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:50.997852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:51.006166 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-12333ab4-10c9-4f34-af94-a34c84192663 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1502 microversion: 2.1 time: 0.185267 Sep 18 00:58:51.007891 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 111/222] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:50 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1502 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:51.032962 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:51.033577 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:51.040632 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:51.042057 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:51.042586 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:51.043355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:51.043881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:51.044369 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:51.179038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:51.186846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-81ed89ad-8afc-4d0a-9f98-5915b5bc96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1516 microversion: 2.1 time: 0.157759 Sep 18 00:58:51.187443 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 112/223] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:51 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1516 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:54.719773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:58:54.721972 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:54.730347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:54.730943 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:54.731345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:54.732075 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:54.732637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:54.733066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:54.756775 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lazy-loading 'flavor' on Instance uuid ae00028e-df3f-4c20-84a1-e0e14f4cc457 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 00:58:54.836664 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2c116578-4987-4bb5-a3a1-2b81e4991949 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457/action" status: 200 len: 32 microversion: 2.1 time: 0.120808 Sep 18 00:58:54.837400 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 112/224] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 00:58:54 2022] POST /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457/action => generated 32 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 00:58:54.850799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:54.850799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:54.857160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:54.857713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:54.858154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:54.859005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:54.859506 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:54.859981 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:55.107751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:55.132551 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9cb8b67a-ab95-4333-a5c0-e3c53d9a27ed tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.289445 Sep 18 00:58:55.134026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 113/225] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:54 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:56.158945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:56.160480 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:56.167743 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:56.168646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:56.169443 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:56.170351 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:56.171829 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:56.175644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:56.328497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:56.336285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-88436f41-41bf-41d4-9704-3350348cecd1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.182824 Sep 18 00:58:56.336855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 113/226] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:56 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:57.359174 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:57.359788 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:57.365541 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:57.366032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:57.366562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:57.367180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:57.367630 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:57.368050 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:57.510481 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:57.524953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8bf37aa2-4318-4840-9432-1be605cf7cb8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.169366 Sep 18 00:58:57.534853 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 114/227] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:57 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:58.549611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:58.550223 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:58.556630 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:58.557104 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:58.557537 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:58.558218 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:58.558722 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:58.559143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:58.688591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:58.696422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8672fc47-87e8-4dd7-8b2b-74e446c94fd9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.150669 Sep 18 00:58:58.696992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 114/228] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:58 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:58:59.719876 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:58:59.720472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:58:59.728077 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:59.728566 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:59.728995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:59.729682 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:58:59.730130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:58:59.730582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:58:59.867750 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:58:59.876447 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b6b03f7a-518c-4f4b-978d-977d1d9c9a74 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.160418 Sep 18 00:58:59.877039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 115/229] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:58:59 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:00.902434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:00.903283 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:00.910850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:00.911486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:00.912256 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:00.913027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:00.913573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:00.914037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:01.125623 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:01.139323 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ed74f98b-19bb-4b5d-930b-1f4d597296cd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.241908 Sep 18 00:59:01.151166 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 115/230] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:00 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:02.173100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:02.174201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:02.182927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:02.183459 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:02.183942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:02.184896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:02.185336 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:02.185761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:02.350176 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:02.359068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0221e06c-b872-4a8c-be00-9b30beb9845f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.189955 Sep 18 00:59:02.367038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 116/231] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:02 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:03.383508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:03.384965 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:03.393025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:03.393794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:03.394438 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:03.395358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:03.396604 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:03.397655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:03.563721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:03.573648 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-abfd64be-7368-4b3a-91f9-45d1e6be2ca2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.194378 Sep 18 00:59:03.574505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 116/232] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:03 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:04.598209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:04.599435 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:04.601343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=26 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:04.606860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:04.607350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:04.607780 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:04.608466 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:04.608917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:04.609396 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:04.781060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:04.789514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f230377e-14ef-46d3-a086-d20c89d791ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.195570 Sep 18 00:59:04.790274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 117/233] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:04 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:05.812684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:05.813299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:05.816166 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=26 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:05.822302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:05.822747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:05.823191 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:05.824157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:05.824609 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:05.825084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:05.966392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:05.974706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8b8bb45d-8d40-4c73-8835-739302e1d5e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.165537 Sep 18 00:59:05.975274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 117/234] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:05 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:07.001414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:07.002440 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:07.017864 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:07.018863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:07.020025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:07.021575 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:07.022485 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:07.023543 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:07.200514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:07.208478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e706a4c9-3c36-46d2-86ea-b4d464eac7fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.214221 Sep 18 00:59:07.208846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 118/235] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:06 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:08.231178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:08.232535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:08.240581 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:08.241167 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:08.241971 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:08.243137 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:08.244288 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:08.245063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:08.434503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:08.444497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bcfd4560-6fc3-461d-86dd-e84abb26ad0a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.215595 Sep 18 00:59:08.444497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 118/236] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:08 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:09.467251 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:09.468009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:09.474597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:09.475077 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:09.475507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:09.476187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:09.476627 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:09.477037 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:09.632095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:09.640077 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0581e103-ad49-445a-9a1e-1079e59745bf tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.176889 Sep 18 00:59:09.640644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 119/237] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:09 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:09.756971 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:09.757489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:09.763889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:09.764368 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:09.764947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:09.765646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:09.766091 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:09.766533 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:09.792406 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a1c2bfc8-b411-491c-b2eb-fcd982ac5596 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/ips" status: 200 len: 152 microversion: 2.20 time: 0.038716 Sep 18 00:59:09.792897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 119/238] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Sep 18 00:59:09 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/ips => generated 152 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 00:59:10.666159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:10.667435 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:10.675533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:10.676249 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:10.676941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:10.677989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:10.678826 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:10.679475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:10.851554 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:10.860801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-61c949a5-002e-43d6-a7a5-9e75fc0e0bb4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.201037 Sep 18 00:59:10.862588 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 120/239] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:10 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:11.887278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:11.888216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:11.895616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:11.896657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:11.897157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:11.898055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:11.898584 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:11.899080 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:12.087048 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:12.099885 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a9e223fd-8dcf-4b02-8771-ee218d8642d5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.216321 Sep 18 00:59:12.100752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 120/240] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:11 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:12.746071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:12.746677 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:12.753195 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:12.753737 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:12.754220 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:12.755077 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:12.755525 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:12.755981 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:12.779799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lazy-loading 'flavor' on Instance uuid 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 00:59:13.123600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:13.124193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:13.131487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:13.132130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:13.132874 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:13.137535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:13.137535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:13.137535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:13.284361 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:13.292569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5536fcb7-300a-4d45-a8a4-2a22d28acd41 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.172415 Sep 18 00:59:13.293140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 121/241] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:13 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:14.315525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:14.316227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:14.322600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:14.323076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:14.323502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:14.324450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:14.324895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:14.325312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:14.475123 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:14.483254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b4f535ef-6762-42bc-b12e-949c34d2006b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.171644 Sep 18 00:59:14.483824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 122/242] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:14 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:15.512338 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:15.513414 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:15.527339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:15.527844 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:15.528282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:15.529014 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:15.529459 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:15.531010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:15.676319 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:15.684654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a4777532-2662-410e-869c-7c946a83d9e1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.180818 Sep 18 00:59:15.685222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 123/243] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:15 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:16.708517 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:16.710121 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:16.720827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:16.721620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:16.722330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:16.723491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:16.724245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:16.725502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:16.879843 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:16.890331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2b8f0be7-9670-4974-80c0-624caf17ef6a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.186182 Sep 18 00:59:16.891018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 124/244] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:16 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:17.915338 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:17.916719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:17.923546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:17.924050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:17.924484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:17.925422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:17.925930 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:17.926312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:17.936097 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:18.259293 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:18.267671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3ee50724-bc8b-4053-9931-b0a959c2904d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.356342 Sep 18 00:59:18.268524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 125/245] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:17 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:18.507371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a73f7d23-0217-4b01-86b8-97847c57c2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/action" status: 202 len: 0 microversion: 2.20 time: 5.765417 Sep 18 00:59:18.511434 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 121/246] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 18 00:59:12 2022] POST /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/action => generated 0 bytes in 5770 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 00:59:18.521356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:18.530866 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:18.538157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:18.540555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:18.540555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:18.541063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:18.541515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:18.541933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:18.685070 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:18.694615 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:18.696349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:18.697502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-361dce56-6f40-453c-b53a-dd7cb59ef9f3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.181827 Sep 18 00:59:18.698060 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 126/247] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:18 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:19.293948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:19.294670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:19.302040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:19.303019 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:19.303928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:19.304693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:19.305334 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:19.305844 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:19.310536 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:19.449939 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:19.457999 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3f5b170d-d5d6-41c0-9d51-09d528ed56c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.168230 Sep 18 00:59:19.458614 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 122/248] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:19 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:19.722849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:19.723365 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:19.729838 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:19.730327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:19.730798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:19.731507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:19.731962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:19.732387 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:20.152853 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:20.162143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:20.163360 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:20.164540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6754bf28-98de-4f90-8088-aac39d5d245a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.445755 Sep 18 00:59:20.165088 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 127/249] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:19 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:20.481375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:20.481991 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:20.488428 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:20.488954 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:20.489397 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:20.490693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:20.491488 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:20.492244 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:20.655852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:20.655852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-21debcac-25f3-4894-8615-51427fbabda0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.176086 Sep 18 00:59:20.655852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 123/250] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:20 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:21.200608 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:21.201823 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:21.208250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.209439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.209894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:21.210600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.211053 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.211477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:21.399773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:21.409724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:21.411284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:21.412677 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-14c6e818-00cd-4bc5-b63d-b03013e04c0a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.220769 Sep 18 00:59:21.413322 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 128/251] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:21 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:21.677285 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:21.678381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:21.685502 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.685995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.686431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:21.687161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.687609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.688701 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:21.841427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:21.850698 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c4071487-c1e2-4c67-bc2a-cc3c474e5db2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1502 microversion: 2.1 time: 0.177394 Sep 18 00:59:21.851356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 124/252] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:21 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1502 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:21.874049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "601b7c01-f681-4054-ad5b-85097408907d"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:21.876454 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:21.882540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.883032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.883501 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:21.884201 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.884655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.885084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:21.909326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] HTTP exception thrown: Cannot 'attach_volume' instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 while it is in vm_state rescued Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Request method failure captured: Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: POST /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457/os-volume_attachments HTTP/1.1 Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 74 Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzhkxRTrCUQQ4CHYpp5ib4E3J8zJRLpaPqCNiJzmchcAu_ipeHd4RwYdO1mlchW4uUJ4qXEZ8cWy5sMB5AwukkrGJE4mZR_1pkUaRSmGbYTXpM7QzG3OSBhI3tMi7AEO_3YHBoAPk5T6bb6rWvo2L-te49vgOqSNbUeL_AMq01aAlA Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 00:59:21.911072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_df01daf86854497fad1d1d016c269ef1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}] Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 54d4e437036640b58fda503c4d213f5c Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {"volumeAttachment": {"volumeId": "601b7c01-f681-4054-ad5b-85097408907d"}} Sep 18 00:59:21.913232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Cannot 'attach_volume' instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 while it is in vm_state rescued Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'server_id': 'ae00028e-df3f-4c20-84a1-e0e14f4cc457', 'body': {'volumeAttachment': {'volumeId': '601b7c01-f681-4054-ad5b-85097408907d'}}} Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 00:59:21.914393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 while it is in vm_state rescued Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 00:59:21.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 00:59:21.916994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 00:59:21.916994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 00:59:21.916994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 while it is in vm_state rescued Sep 18 00:59:21.916994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 00:59:21.916994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Returning 409 to user: Cannot 'attach_volume' instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 while it is in vm_state rescued {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 00:59:21.916994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-38823d22-1307-4b0c-97e6-22beb9ce2993 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.043167 Sep 18 00:59:21.916994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 129/253] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 00:59:21 2022] POST /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457/os-volume_attachments => generated 152 bytes in 44 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 18 00:59:21.923040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:21.923612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:21.930495 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.931631 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.932234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:21.933032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:21.933516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:21.933943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:22.019672 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b229cf39-6273-4cdc-999d-b153c968a6e6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457/action" status: 202 len: 0 microversion: 2.1 time: 0.099861 Sep 18 00:59:22.020339 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 125/254] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 00:59:21 2022] POST /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457/action => generated 0 bytes in 102 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 00:59:22.029795 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:22.030639 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:22.041105 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:22.041675 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:22.042166 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:22.042937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:22.043390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:22.044004 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:22.271392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:22.280187 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7a830825-ef47-430a-9a72-894b9fd6f0ac tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1510 microversion: 2.1 time: 0.255317 Sep 18 00:59:22.281357 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 130/255] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:22 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1510 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:22.441575 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:22.442391 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:22.449241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:22.449998 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:22.450433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:22.451178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:22.452494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:22.452923 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:22.679549 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:22.692439 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:22.695751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:22.696875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-593de8bd-67d9-4197-9e2b-8c731a9e48c8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.260493 Sep 18 00:59:22.697437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 126/256] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:22 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:23.304865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:23.305684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:23.313657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:23.314355 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:23.314999 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:23.315950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:23.316503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:23.317012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:23.466189 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:23.474940 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2a281937-2628-45d9-973a-5274fe77d1da tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1510 microversion: 2.1 time: 0.174619 Sep 18 00:59:23.475509 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 131/257] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:23 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1510 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:23.726144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:23.726783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:23.732706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:23.733188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:23.733622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:23.734321 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:23.734784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:23.735206 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:23.890222 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:23.898925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:23.900376 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:23.901524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-58a53cd0-919b-4281-835e-0b856d82c6bb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.179807 Sep 18 00:59:23.902410 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 127/258] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:23 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:24.409790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:24.410377 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:24.416733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:24.417206 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:24.417638 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:24.418322 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:24.418854 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:24.419238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:24.442595 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-099affe7-a472-4297-9185-4c2db5720cb8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/ips" status: 200 len: 147 microversion: 2.1 time: 0.036611 Sep 18 00:59:24.443677 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 132/259] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 00:59:24 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/ips => generated 147 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:24.500463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:24.501337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:24.512060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:24.513560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:24.514296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:24.515447 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:24.516181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:24.517143 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:24.679776 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:24.688467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-49a75399-20f7-4eb1-a280-ec98629421a1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1510 microversion: 2.1 time: 0.194642 Sep 18 00:59:24.689095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 128/260] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:24 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1510 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:24.928174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:24.928702 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:24.935016 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:24.935492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:24.935917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:24.936592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:24.937039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:24.937453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:25.064326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:25.073026 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:25.074781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:25.075927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-72987c3a-4e45-4348-848d-5715fd902a75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.152185 Sep 18 00:59:25.076467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 133/261] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:24 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:25.713081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:25.713687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:25.720218 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:25.720703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:25.721134 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:25.721836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:25.722283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:25.722735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:25.858002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:25.866784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cc4e7f39-12fd-4c94-8634-53ccc201f6d7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1516 microversion: 2.1 time: 0.158102 Sep 18 00:59:25.867353 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 129/262] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:25 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1516 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:26.102781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:26.103901 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:26.112031 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:26.112835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:26.113556 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:26.114790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:26.116137 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:26.117371 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:26.291461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:26.300154 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:26.301621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:26.303292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a98db500-aa12-4ac9-badb-04bd654f8dd2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.206173 Sep 18 00:59:26.303868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 134/263] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:26 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:27.328498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:27.329081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:27.335861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:27.336344 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:27.336783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:27.337476 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:27.337927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:27.338343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:27.467240 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:27.475227 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:27.476696 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:27.477835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-23e9d7c6-20f0-435f-bb81-3b477e9afb70 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.153482 Sep 18 00:59:27.478386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 130/264] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:27 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:28.502483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:28.503025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:28.512486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:28.512980 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:28.513413 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:28.514128 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:28.514616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:28.515037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:28.647571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:28.655729 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:28.657149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:28.658294 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8a1060ad-298c-4713-8483-b075025d4400 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.160018 Sep 18 00:59:28.658868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 135/265] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:28 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:29.683535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:29.684104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:29.689967 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:29.690447 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:29.690913 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:29.691593 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:29.692043 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:29.692465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:29.818972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:29.828703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:29.830163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:29.831354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c80d56e4-3dfa-42c8-b033-929d08847a26 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.151771 Sep 18 00:59:29.831914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 131/266] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:29 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:30.104302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0838177b-6c5f-497e-807d-d10c52962e00"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:30.106697 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:30.113137 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:30.113616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:30.114049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:30.114873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:30.115229 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:30.115652 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:30.416634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8fa2df68-8e70-442e-bcae-1a569cf9636a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.316232 Sep 18 00:59:30.417245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 136/267] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 00:59:30 2022] POST /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 194 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:30.476861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-044a6ace-996b-4689-b0a3-8486d032e94f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'create', calling method: .version_select at 0x7fcab1d96280>, body: {"keypair": {"name": "tempest-keypair-1235240290", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL4Ct//QB2QT8Aqzr1lpNhHmemZrIIaHNodAHWJBCaa7I2ofMHWsWx5dLT/c1emVhMKPAXdBeynAHq3ag2elGR+KX8fB9IPc99qCgpJY0CK3x6Fv6Ag1w9OJFt6kFcSZfg=="}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:30.504525 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-044a6ace-996b-4689-b0a3-8486d032e94f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for project df01daf86854497fad1d1d016c269ef1. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:59:30.512065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-044a6ace-996b-4689-b0a3-8486d032e94f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for user 54d4e437036640b58fda503c4d213f5c and project df01daf86854497fad1d1d016c269ef1. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:59:30.542908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-044a6ace-996b-4689-b0a3-8486d032e94f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.074338 Sep 18 00:59:30.543784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 132/268] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 00:59:30 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 00:59:30.858332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:30.859127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:30.873090 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:30.874051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:30.874775 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:30.875763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:30.876478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:30.877122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:31.121711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:31.130284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:31.132300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:31.133507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-726e2053-54e6-4c4a-9e84-47f7972c864a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.280691 Sep 18 00:59:31.134072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 137/269] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:30 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:32.096501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1502004691", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a19f83a0-76a4-4dc1-a372-2847ecc4812d"}], "security_groups": [{"name": "tempest-securitygroup--1130715814"}], "key_name": "tempest-keypair-1235240290", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:32.160057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:32.160666 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:32.167444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:32.167916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:32.168346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:32.169051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:32.169505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:32.169923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:32.221498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] validate_networks() for [('a19f83a0-76a4-4dc1-a372-2847ecc4812d', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 00:59:32.409794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:32.423274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:32.425114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:32.426569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-542dc9d7-388b-4d7a-ac43-d88b3e0bfef5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.270302 Sep 18 00:59:32.427893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 138/270] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:32 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:59:32.715843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 00:59:32.716272 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 00:59:32.716770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 00:59:32.717200 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 00:59:32.717647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 00:59:32.739460 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:32.740008 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:32.740426 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:32.741402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:32.741915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:32.742342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:32.760542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for project df01daf86854497fad1d1d016c269ef1. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:59:32.764878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Getting quotas for user 54d4e437036640b58fda503c4d213f5c and project df01daf86854497fad1d1d016c269ef1. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:59:32.776703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 00:59:32.779117 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 00:59:32.828611 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d483066f-0a6c-45c7-9a0a-097fedb1c1c1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.736858 Sep 18 00:59:32.829571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 133/271] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 00:59:32 2022] POST /compute/v2.1/servers => generated 409 bytes in 738 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 00:59:32.841023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-40e6c273-7228-40f1-a3b7-b61d30e13159 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:32.841642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-40e6c273-7228-40f1-a3b7-b61d30e13159 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:32.912887 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-40e6c273-7228-40f1-a3b7-b61d30e13159 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:32.922364 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-40e6c273-7228-40f1-a3b7-b61d30e13159 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1246 microversion: 2.1 time: 0.085077 Sep 18 00:59:32.923188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 139/272] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:32 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1246 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:33.462710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:33.463293 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:33.471046 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:33.471741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:33.473474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:33.475152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:33.475722 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:33.476593 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:33.626236 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:33.636190 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:33.637660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:33.638912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1ac75601-d360-4923-ad13-f6daf80f275c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.181094 Sep 18 00:59:33.639465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 134/273] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:33 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:33.948586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:33.949358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:33.957098 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:33.957580 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:33.958014 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:33.958894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:33.959348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:33.959662 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:34.020080 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:34.028477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1d5275a9-a991-4eb4-a21a-ff01f416392b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1250 microversion: 2.1 time: 0.085035 Sep 18 00:59:34.029063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 140/274] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:33 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1250 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:34.666032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:34.666666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:34.668735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=58,nova_api:INSERT=4 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:34.674356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:34.674882 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:34.675312 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:34.676017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:34.676553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:34.676977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:34.823301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:34.831989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:34.833441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:34.835322 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9d41c090-c0b2-4fc7-9598-6707cecba302 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.173223 Sep 18 00:59:34.835720 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 135/275] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:34 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:35.055297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:35.056281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:35.063342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:35.063822 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:35.064253 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:35.064971 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:35.065412 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:35.065832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:35.128450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:35.136679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cb0fd29a-f6b0-4f9a-836f-5de641d28d5a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1250 microversion: 2.1 time: 0.088010 Sep 18 00:59:35.137433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 141/276] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:35 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1250 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:35.860857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:35.861481 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:35.867394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:35.867876 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:35.868308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:35.868996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:35.869446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:35.869867 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:36.001648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:36.016828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:36.018384 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:36.020320 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f8b241bb-1911-4945-a1b3-e0ea48f9649e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.163025 Sep 18 00:59:36.021196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 136/277] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:35 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:36.159338 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:36.159967 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:36.164222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=52 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:36.167750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:36.168442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:36.168876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:36.169570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:36.170019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:36.170433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:36.175311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:36.240481 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:36.248320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2a589513-a0a5-4f9a-9d47-06ac779877b1 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1242 microversion: 2.1 time: 0.092748 Sep 18 00:59:36.248882 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 142/278] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:36 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1242 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:37.051521 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:37.052155 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:37.064716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:37.065333 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:37.065883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:37.066870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:37.067558 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:37.068126 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:37.260204 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:37.268630 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:37.270091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:37.271304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-39823482-6ca0-4891-bfad-4ec8ed4b8398 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1745 microversion: 2.20 time: 0.223617 Sep 18 00:59:37.272008 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 137/279] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:37 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1745 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:37.272135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:37.273243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:37.279713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:37.280199 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:37.280634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:37.281326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:37.281785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:37.282208 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:37.363031 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:37.371500 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-372f67a6-feb4-4606-9a2c-be06b94dab20 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1304 microversion: 2.1 time: 0.103186 Sep 18 00:59:37.372101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 143/280] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:37 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1304 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:38.300996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:38.303110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:38.320028 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:38.326044 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:38.327658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:38.331202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:38.331844 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:38.332638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:38.394658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:38.395395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:38.409631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:38.410410 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:38.411254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:38.415423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:38.415913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:38.416404 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:38.918365 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:38.932849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7537bf5f-1186-4525-8f86-c4c176c28fe5 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.542398 Sep 18 00:59:38.934143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 144/281] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:38 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:38.975323 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:38.984999 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:38.986342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:38.988061 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-81b48cad-6c5d-45ae-b9d5-20e92d0f778b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1745 microversion: 2.20 time: 0.694429 Sep 18 00:59:38.988898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 138/282] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:38 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1745 bytes in 696 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:39.288890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c0d415a0-d760-4444-8c7d-4d309abbec74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: .version_select at 0x7fcab1f25940>, body: {"keypair": {"name": "tempest-keypair-467033041", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIwhGsyOLRoLF2qpmkYgbmjG+RLUqcTAE2Nmp8uamxRimKW3XpDb1gDJqunG8V/HwpmVD3z76JFKPOI5gC+NPFgnNqcHDGvbJPlmovMOqAFCtyLMMPOG7S6Rv7UzC6VB7Q=="}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:39.316878 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c0d415a0-d760-4444-8c7d-4d309abbec74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:59:39.320988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c0d415a0-d760-4444-8c7d-4d309abbec74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:59:39.368735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c0d415a0-d760-4444-8c7d-4d309abbec74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.084245 Sep 18 00:59:39.369891 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 145/283] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 00:59:39 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:39.407923 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-2ec97b67-83b3-41be-99f4-1b0f49a147e2 req-a5d04828-b94c-4830-b823-f15a12904f1d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e1c6d6a-9178-4c19-a032-ef79295fce8c", "tag": "bfcd0e79-9fb2-4117-b12a-c4de2d65b667"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:39.423783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-2ec97b67-83b3-41be-99f4-1b0f49a147e2 req-a5d04828-b94c-4830-b823-f15a12904f1d service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:39.425058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-2ec97b67-83b3-41be-99f4-1b0f49a147e2 req-a5d04828-b94c-4830-b823-f15a12904f1d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:39.425659 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-2ec97b67-83b3-41be-99f4-1b0f49a147e2 req-a5d04828-b94c-4830-b823-f15a12904f1d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:39.440994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-2ec97b67-83b3-41be-99f4-1b0f49a147e2 req-a5d04828-b94c-4830-b823-f15a12904f1d service nova] Creating event network-changed:bfcd0e79-9fb2-4117-b12a-c4de2d65b667 for instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c on hv-cinder-maste Sep 18 00:59:39.447956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-2ec97b67-83b3-41be-99f4-1b0f49a147e2 req-a5d04828-b94c-4830-b823-f15a12904f1d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044823 Sep 18 00:59:39.449176 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 139/284] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:59:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:39.963410 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:39.964168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:39.973396 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:39.974326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:39.974961 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:39.976052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:39.976781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:39.977118 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:40.015879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:40.016903 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:40.026155 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:40.026510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:40.026958 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:40.027863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:40.028114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:40.039619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:40.046970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:40.286375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:40.307383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cef89422-3337-4d00-b5a2-24c7648f80f7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.340115 Sep 18 00:59:40.308299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 146/285] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:39 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:40.352950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:40.365029 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:40.367436 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:40.369048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-57431ce2-d1fa-431f-9633-fffbe0672b7b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1745 microversion: 2.20 time: 0.357427 Sep 18 00:59:40.369953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 140/286] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:40 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1745 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:40.420419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-29edff23-b7ee-4631-9cbf-0f2d74e6394a req-a3ca7955-d1de-491b-bf15-6976274631c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5efa25bd-04ff-49f6-9fd5-6e8c82bcd152", "name": "network-vif-plugged", "status": "completed", "tag": "af5eca8e-a823-4591-9077-2a4b9b7fb155"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:40.431472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-29edff23-b7ee-4631-9cbf-0f2d74e6394a req-a3ca7955-d1de-491b-bf15-6976274631c6 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:40.432094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-29edff23-b7ee-4631-9cbf-0f2d74e6394a req-a3ca7955-d1de-491b-bf15-6976274631c6 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:40.432407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-29edff23-b7ee-4631-9cbf-0f2d74e6394a req-a3ca7955-d1de-491b-bf15-6976274631c6 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:40.452301 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-29edff23-b7ee-4631-9cbf-0f2d74e6394a req-a3ca7955-d1de-491b-bf15-6976274631c6 service nova] Creating event network-vif-plugged:af5eca8e-a823-4591-9077-2a4b9b7fb155 for instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 on hv-cinder-maste Sep 18 00:59:40.458488 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-29edff23-b7ee-4631-9cbf-0f2d74e6394a req-a3ca7955-d1de-491b-bf15-6976274631c6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041613 Sep 18 00:59:40.459383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 147/287] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:59:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:41.334140 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:41.337770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:41.356477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:41.357072 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:41.358080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:41.360247 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:41.360837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:41.361206 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:41.416911 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:41.417630 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:41.431501 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:41.435617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:41.436567 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:41.438611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:41.439135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:41.440065 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:41.831311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:41.841046 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-34608948-97e6-4e84-866d-8bda74a4a20f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.513307 Sep 18 00:59:41.841760 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 141/288] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:41 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:41.856868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:41.872205 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2051745844", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3227bd19-4c4c-45c0-aee5-e73b3870bae1"}], "security_groups": [{"name": "tempest-securitygroup--1642669758"}], "key_name": "tempest-keypair-467033041", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:41.872860 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:41.874702 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:41.880058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ebfc8105-10cb-44b2-9d67-39a9fd60221d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.484433 Sep 18 00:59:41.881185 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 148/289] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:41 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:42.212754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] validate_networks() for [('3227bd19-4c4c-45c0-aee5-e73b3870bae1', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 00:59:42.750197 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:42.771584 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=10 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:42.868789 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:42.869480 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:42.876990 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:42.877755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:42.878754 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:42.879791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:42.880417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:42.881010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:43.164573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 00:59:43.165056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 00:59:43.165858 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 00:59:43.166403 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 00:59:43.167154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 00:59:43.197692 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:43.211915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:43.212415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:43.212914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:43.213493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f69d57e4-edf2-48e1-9854-be41263abe54 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.351152 Sep 18 00:59:43.214111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:43.214858 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:43.215781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:43.226893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 149/290] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:42 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:43.234279 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:43.235116 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:43.242620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:43.243365 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:43.243966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:43.244855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:43.245456 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:43.246200 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:43.256801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 00:59:43.262486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 00:59:43.277104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 00:59:43.279682 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 00:59:43.377294 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e1af52de-f1d5-488c-8f6e-1680628faf48 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.531475 Sep 18 00:59:43.378331 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 142/291] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 00:59:41 2022] POST /compute/v2.1/servers => generated 402 bytes in 1536 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 00:59:43.390589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-13a8d1f6-0cf9-47e9-9dad-a28ec3b43d68 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:43.392892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-13a8d1f6-0cf9-47e9-9dad-a28ec3b43d68 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:43.551755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-13a8d1f6-0cf9-47e9-9dad-a28ec3b43d68 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:43.563663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-13a8d1f6-0cf9-47e9-9dad-a28ec3b43d68 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1241 microversion: 2.1 time: 0.173243 Sep 18 00:59:43.579354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 143/292] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:43 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1241 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:43.611826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:43.623701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:43.626432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:43.628657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-dc39e11c-af8e-4fa4-ac2c-03364d55a261 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.400444 Sep 18 00:59:43.641291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 150/293] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:43 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:44.238644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:44.239898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:44.252516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:44.257023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:44.257634 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:44.258570 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:44.261450 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:44.262082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:44.604177 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:44.605441 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:44.613809 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:44.614444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:44.626425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:44.627545 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:44.628163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:44.628612 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:44.877118 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:44.881034 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:44.892634 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-76202c79-1d17-4016-b79c-13e40772b421 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.658231 Sep 18 00:59:44.893309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 144/294] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:44 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:44.910371 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-54a9424e-99f0-46d3-bad5-e1d9fc087df1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1237 microversion: 2.1 time: 0.310420 Sep 18 00:59:44.912851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 151/295] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:44 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1237 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 00:59:44.919542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:44.931779 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:44.956768 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:44.957286 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:44.957763 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:44.958498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:44.959054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:44.959558 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:45.304737 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:45.349992 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:45.356382 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:45.381034 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b60ffda7-a8d6-438b-82f2-2328e01c0924 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.468888 Sep 18 00:59:45.387009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 145/296] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:44 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:45.919560 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:45.921961 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:45.937214 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:45.938202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:45.953102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:45.954307 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:45.957643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:45.958533 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:45.960850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:45.961122 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:45.962240 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:45.963210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:45.964350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:45.965674 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:45.966349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:45.968673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:46.298693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:46.313324 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-780b73a3-98e2-4c85-928f-1edd9d140f4d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1311 microversion: 2.1 time: 0.381072 Sep 18 00:59:46.323266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 146/297] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:45 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1311 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:46.419828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:46.420612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:46.440401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:46.445950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:46.448756 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:46.450710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:46.452326 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:46.452895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:46.724262 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:46.758062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5cc00e81-16c0-4a13-862c-6a7af337117a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.842469 Sep 18 00:59:46.761974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 152/298] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:45 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:46.831373 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:46.850972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:46.861081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:46.866076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b0b57915-81a7-4b7c-a38f-7b3d0ad965e3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.451367 Sep 18 00:59:46.887898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 147/299] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:46 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:59:47.357855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:47.359763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:47.390473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:47.391747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:47.393201 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:47.394706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:47.396594 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:47.397398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:47.692223 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:47.712240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1a0de5eb-041d-4e35-baea-b42516af1675 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1299 microversion: 2.1 time: 0.366872 Sep 18 00:59:47.713164 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 153/300] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:47 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1299 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:47.786636 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:47.788814 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:47.800918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:47.803332 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:47.804605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:47.808565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:47.810123 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:47.816139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:47.902341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:47.903375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:47.917844 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:47.922047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:47.929616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:47.930607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:47.931094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:47.931528 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:48.728621 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:48.740074 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1952039a-4bab-4971-b21b-9549004a306e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.958021 Sep 18 00:59:48.740656 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 148/301] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:47 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 959 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:48.755213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-e09ac45d-4598-42ae-b59a-afba1d609168 req-b0db268b-67ee-4ab1-9506-e095c1c62b1d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e1c6d6a-9178-4c19-a032-ef79295fce8c", "name": "network-vif-plugged", "status": "completed", "tag": "bfcd0e79-9fb2-4117-b12a-c4de2d65b667"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:48.777872 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e09ac45d-4598-42ae-b59a-afba1d609168 req-b0db268b-67ee-4ab1-9506-e095c1c62b1d service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:48.786056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e09ac45d-4598-42ae-b59a-afba1d609168 req-b0db268b-67ee-4ab1-9506-e095c1c62b1d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:48.786728 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e09ac45d-4598-42ae-b59a-afba1d609168 req-b0db268b-67ee-4ab1-9506-e095c1c62b1d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:48.796764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:48.813195 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:48.815821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-e09ac45d-4598-42ae-b59a-afba1d609168 req-b0db268b-67ee-4ab1-9506-e095c1c62b1d service nova] Creating event network-vif-plugged:bfcd0e79-9fb2-4117-b12a-c4de2d65b667 for instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c on hv-cinder-maste Sep 18 00:59:48.818435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:48.823021 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-89afce47-ab96-407e-ba53-e1091cc06fe7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.925412 Sep 18 00:59:48.824220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 154/302] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:47 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 929 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:48.830267 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-e09ac45d-4598-42ae-b59a-afba1d609168 req-b0db268b-67ee-4ab1-9506-e095c1c62b1d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087640 Sep 18 00:59:48.831291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 149/303] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:59:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:48.843511 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:48.844141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:48.857635 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:48.859766 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:48.862311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:48.863372 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:48.868417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:48.869275 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:49.332526 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:49.473014 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:49.492694 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-233565fc-4a7f-41ed-822c-a62ae4fd8b86 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.661772 Sep 18 00:59:49.493257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 155/304] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:48 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:49.771151 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:49.772992 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:49.794442 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:49.799047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:49.801894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:49.805527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:49.806501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:49.811538 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:49.854277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:49.855569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:49.870417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:49.870918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:49.874116 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:49.875064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:49.879258 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:49.879919 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:50.239605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:50.255251 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:50.259169 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:50.264055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8258991e-f8f6-4117-b4b8-cec9a7679a10 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.415819 Sep 18 00:59:50.265802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 156/305] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:49 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:50.268031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:50.283849 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b6978ba1-c328-4055-ad61-db390ce872a8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1372 microversion: 2.1 time: 0.516882 Sep 18 00:59:50.302103 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 150/306] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:49 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1372 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:50.436646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-5530cab6-fbb5-428c-a782-1da0005da50d req-32db89de-1ecb-4939-9e35-c27ccb5ff560 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1a2de33a-66be-4df0-9c70-adf5c1b03ec8", "tag": "b8d36f12-54c1-44a9-97c9-bfd9768a810e"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:50.456712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5530cab6-fbb5-428c-a782-1da0005da50d req-32db89de-1ecb-4939-9e35-c27ccb5ff560 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:50.459474 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5530cab6-fbb5-428c-a782-1da0005da50d req-32db89de-1ecb-4939-9e35-c27ccb5ff560 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:50.460633 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5530cab6-fbb5-428c-a782-1da0005da50d req-32db89de-1ecb-4939-9e35-c27ccb5ff560 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:50.488328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-5530cab6-fbb5-428c-a782-1da0005da50d req-32db89de-1ecb-4939-9e35-c27ccb5ff560 service nova] Creating event network-changed:b8d36f12-54c1-44a9-97c9-bfd9768a810e for instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 on hv-cinder-maste Sep 18 00:59:50.496702 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-5530cab6-fbb5-428c-a782-1da0005da50d req-32db89de-1ecb-4939-9e35-c27ccb5ff560 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063681 Sep 18 00:59:50.500276 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 157/307] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:59:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:50.516428 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:50.517033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:50.525451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:50.534750 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:50.535462 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:50.536949 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:50.537633 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:50.538841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:50.763380 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:50.774789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-81e37d81-90cc-4795-8b1b-0a3db132e614 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.261999 Sep 18 00:59:50.776413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 151/308] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:50 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:51.295636 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:51.296434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:51.321267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:51.323208 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:51.323831 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:51.325170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:51.326847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:51.330770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:51.333543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:51.334837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:51.338000 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:51.338798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:51.340346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:51.341278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:51.343239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:51.347125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:51.855859 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:51.870019 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-19e70047-7255-42b0-9542-8674a1db3661 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1566 microversion: 2.1 time: 0.552318 Sep 18 00:59:51.875177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 152/309] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:51 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1566 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:51.900398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:51.901307 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:51.916799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:51.923570 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:51.926725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:51.927965 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:51.929527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:51.931949 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:51.978984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:51.994292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:52.000145 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:52.001634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-53f3557c-464a-4106-93a3-421d9062b8b9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.712070 Sep 18 00:59:52.002539 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 158/310] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:51 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 714 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:52.007340 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:52.016510 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:52.032298 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:52.034932 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:52.036076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:52.038546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:52.046388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:52.048226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:52.481679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b2e2a998-ac01-4727-b083-def5eeed9c9d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.474723 Sep 18 00:59:52.484530 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:52.485696 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 159/311] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 00:59:52 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-interface => generated 284 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:52.494609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a5d987b0-d0ff-41a3-81db-91f8fc51e2b9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.618057 Sep 18 00:59:52.495185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 153/312] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:51 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:53.031683 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:53.032302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:53.047736 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:53.048524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:53.049883 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:53.055282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:53.057921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:53.061648 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:53.520448 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:53.521703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:53.525588 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:53.561702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:53.563060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:53.564633 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:53.570115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:53.576180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:53.578607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:53.582359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=8 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:54.389125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:54.398286 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:54.399774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:54.412148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ed847817-d2ca-4212-b885-9893b001b1c5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 1.384493 Sep 18 00:59:54.416957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 160/313] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:53 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 1390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 00:59:54.633616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:54.652963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a2f8ce69-7c8c-4424-bb2d-0df2055c55c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 1.136526 Sep 18 00:59:54.653437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 154/314] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:53 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 1138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:55.451485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:55.452746 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:55.466024 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:55.472272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:55.481055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:55.481846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:55.482484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:55.482936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:55.574586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 00:59:55.684933 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:55.687168 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:55.714956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:55.714956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:55.714956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:55.714956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:55.715620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:55.715620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:56.281619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:56.298705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5ed14fb3-3e99-452a-b8d4-53bb23d4bd75 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.617932 Sep 18 00:59:56.318966 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 155/315] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:55 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:56.506143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:56.520807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:56.527100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:56.530429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-22f5fcdc-b35b-4260-bf66-4623c0054c75 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 1.081770 Sep 18 00:59:56.531291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 161/316] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:55 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 1085 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:57.326762 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:57.327372 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:57.333567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:57.345156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:57.346051 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:57.349667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:57.353712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:57.357443 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:57.560433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:57.562334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:57.568923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:57.569654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:57.570244 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:57.571140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:57.571774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:57.572344 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:58.265392 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:58.283706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-484ba5b0-4a31-4a2f-b74d-3ba4f4ff4b80 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.959648 Sep 18 00:59:58.285688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 156/317] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:57 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 963 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 00:59:58.308636 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-0597266e-d460-4379-80e7-5488875c02a1 req-7d33d8ee-5fd0-4ab3-8ffd-1f0684466c21 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e1c6d6a-9178-4c19-a032-ef79295fce8c", "tag": "bfcd0e79-9fb2-4117-b12a-c4de2d65b667"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 00:59:58.403879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-0597266e-d460-4379-80e7-5488875c02a1 req-7d33d8ee-5fd0-4ab3-8ffd-1f0684466c21 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:58.416232 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-0597266e-d460-4379-80e7-5488875c02a1 req-7d33d8ee-5fd0-4ab3-8ffd-1f0684466c21 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.023s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:58.426148 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-0597266e-d460-4379-80e7-5488875c02a1 req-7d33d8ee-5fd0-4ab3-8ffd-1f0684466c21 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:58.496406 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-0597266e-d460-4379-80e7-5488875c02a1 req-7d33d8ee-5fd0-4ab3-8ffd-1f0684466c21 service nova] Creating event network-changed:bfcd0e79-9fb2-4117-b12a-c4de2d65b667 for instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c on hv-cinder-maste Sep 18 00:59:58.545888 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-0597266e-d460-4379-80e7-5488875c02a1 req-7d33d8ee-5fd0-4ab3-8ffd-1f0684466c21 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.235106 Sep 18 00:59:58.545888 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 157/318] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 00:59:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 00:59:58.627687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:58.640326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:58.641789 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 00:59:58.655531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fb483f8c-e187-4e81-ae67-ff15e18fb26b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 1.099762 Sep 18 00:59:58.656140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 162/319] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:57 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 1101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 00:59:59.310322 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:59.311243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:59.320397 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:59.323451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:59.324063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:59.324964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:59.327126 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:59.327786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:59.686684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 00:59:59.688435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 00:59:59.705111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:59.708783 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:59.709367 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 00:59:59.711602 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 00:59:59.715061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 00:59:59.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:00.089196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:00.102855 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-00a9e971-25b1-4b42-90ce-ce8576cf9387 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.796416 Sep 18 01:00:00.115428 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 158/320] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 00:59:59 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:00.398966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:00.411844 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:00.415271 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:00.422622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b1ea9288-8f8b-4bc7-a0c5-c028f1424355 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.740351 Sep 18 01:00:00.423711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 163/321] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 00:59:59 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 742 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:01.135237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:01.135846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:01.155064 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:01.155761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:01.156214 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:01.156958 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:01.157417 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:01.157850 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:01.453925 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:01.456938 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:01.469534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:01.472346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:01.472966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:01.476610 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:01.477688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:01.485735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:01.590415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:01.598734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:01.610690 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b0d762f4-116c-4e99-afa9-3af9c41f5d53 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.479931 Sep 18 01:00:01.612034 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 159/322] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:01 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:01.819582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:01.829897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:01.840010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:01.844041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-84f83096-bc38-477d-8f97-13ba0cbdba3d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.396251 Sep 18 01:00:01.847079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 164/323] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:01 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:02.638653 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:02.640654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:02.653842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:02.654697 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:02.656485 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:02.657377 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:02.659172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:02.661521 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:02.878131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:02.879084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:02.890083 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:02.892478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:02.895205 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:02.899710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:02.900334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:02.900781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:03.392370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:03.410067 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4017afcf-f323-44c4-b178-aa67b411fc32 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.773487 Sep 18 01:00:03.414164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 160/324] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:02 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:03.472397 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:03.487461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:03.489789 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:03.496755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bb006456-5e2e-42c6-9f5e-68c2fb4e126e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.623404 Sep 18 01:00:03.500289 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 165/325] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:02 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:04.434416 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:04.435000 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:04.447308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:04.447811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:04.450329 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:04.454862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:04.455352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:04.455789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:04.526126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:04.526978 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:04.545186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:04.546071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:04.546561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:04.547348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:04.557969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:04.558977 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:05.145028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:05.155439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:05.165566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:05.173151 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-702a19d9-030f-4d68-a73c-eaa58e9235b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.652860 Sep 18 01:00:05.179252 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 166/326] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:04 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:05.203021 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:05.215429 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6e6f4e9f-d0c9-4832-b2a1-e37e57a708d9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.783348 Sep 18 01:00:05.217828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 161/327] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:04 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:06.206698 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:06.207328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:06.209977 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:06.221325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:06.221791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:06.242013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:06.243725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:06.246018 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:06.247220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:06.247903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:06.249022 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:06.262550 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:06.269587 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:06.274740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:06.278326 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:06.280649 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:06.283018 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:06.286544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:06.842184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:06.857832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:06.863597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:06.880713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-198eb652-ba03-440c-89d2-adbed914fd4b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.663015 Sep 18 01:00:06.883704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 167/328] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:06 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:07.035688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:07.049488 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6acf3ae5-1338-4bc2-8197-f457f9f18ec2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.806726 Sep 18 01:00:07.050599 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 162/329] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:06 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:07.425611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-187964c2-b75a-43b0-90cc-a5f81c6acbc4 req-3a59fef4-1198-4967-a20c-12b3553ba18a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1a2de33a-66be-4df0-9c70-adf5c1b03ec8", "name": "network-vif-plugged", "status": "completed", "tag": "b8d36f12-54c1-44a9-97c9-bfd9768a810e"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:07.452992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-187964c2-b75a-43b0-90cc-a5f81c6acbc4 req-3a59fef4-1198-4967-a20c-12b3553ba18a service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:07.453501 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-187964c2-b75a-43b0-90cc-a5f81c6acbc4 req-3a59fef4-1198-4967-a20c-12b3553ba18a service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:07.459079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-187964c2-b75a-43b0-90cc-a5f81c6acbc4 req-3a59fef4-1198-4967-a20c-12b3553ba18a service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:07.484667 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-187964c2-b75a-43b0-90cc-a5f81c6acbc4 req-3a59fef4-1198-4967-a20c-12b3553ba18a service nova] Creating event network-vif-plugged:b8d36f12-54c1-44a9-97c9-bfd9768a810e for instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 on hv-cinder-maste Sep 18 01:00:07.498855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-187964c2-b75a-43b0-90cc-a5f81c6acbc4 req-3a59fef4-1198-4967-a20c-12b3553ba18a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090842 Sep 18 01:00:07.499704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 168/330] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:00:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:07.916048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:07.917360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:07.929564 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:07.935329 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:07.936055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:07.936977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:07.937598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:07.938202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:08.080256 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:08.081332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:08.094895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:08.095882 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:08.096489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:08.097538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:08.099334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:08.099923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:08.424117 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:08.438898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:08.443065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:08.447854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-55115bfb-47f4-49dd-81ba-55ec44ff49c2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.537229 Sep 18 01:00:08.462483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 163/331] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:07 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:08.551826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:08.560736 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bc08470e-e5ed-4261-aa1d-9a27ac0e1fe1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.485131 Sep 18 01:00:08.561312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 169/332] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:08 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:09.483729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:09.484344 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:09.503524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:09.506040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:09.506040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:09.506040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:09.517290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:09.517954 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:09.591711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:09.593045 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:09.607348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:09.612912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:09.614430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:09.620886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:09.624673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:09.625385 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:10.064267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:10.077136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-61f9bbbb-92bd-4574-a33d-9fd99043bbf5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1367 microversion: 2.1 time: 0.489171 Sep 18 01:00:10.085580 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 170/333] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:09 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1367 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:10.112572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:10.123957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:10.127849 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:10.129726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2486901a-4997-4ebc-8763-c20f7c8e98ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1740 microversion: 2.20 time: 0.649969 Sep 18 01:00:10.131252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 164/334] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:09 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1740 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:11.117580 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:11.118737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:11.148002 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:11.148751 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:11.153819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:11.155291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:11.158297 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:11.159325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:11.164741 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:11.165651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:11.178251 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:11.178964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:11.179409 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:11.180128 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:11.180585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:11.181007 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:12.594039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:12.614486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:12.622618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:12.631822 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cc337536-9845-4bad-aa14-c31487564c6e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1722 microversion: 2.20 time: 1.473376 Sep 18 01:00:12.633726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 165/335] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:11 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1722 bytes in 1481 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:12.638680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:12.664521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bcc54db8-d21f-4d2d-a12f-83d49b9dad02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1556 microversion: 2.1 time: 1.556234 Sep 18 01:00:12.668488 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 171/336] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:11 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1556 bytes in 1562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:12.695004 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:12.695750 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:12.704414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:12.705549 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:12.706345 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:12.707618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:12.708470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:12.709281 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:12.886614 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-412bd383-8b5b-4a9f-a919-dcd05521caa0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.197094 Sep 18 01:00:12.887552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 166/337] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 01:00:12 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8/os-interface => generated 285 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:13.054350 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-b0c2bcd2-3ca2-431d-b015-640a3836b410 req-f18e3819-f8ee-49f1-b8f2-1ec342025ee8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5efa25bd-04ff-49f6-9fd5-6e8c82bcd152", "name": "network-vif-unplugged", "status": "completed", "tag": "af5eca8e-a823-4591-9077-2a4b9b7fb155"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:13.079193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b0c2bcd2-3ca2-431d-b015-640a3836b410 req-f18e3819-f8ee-49f1-b8f2-1ec342025ee8 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:13.083127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b0c2bcd2-3ca2-431d-b015-640a3836b410 req-f18e3819-f8ee-49f1-b8f2-1ec342025ee8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:13.083833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b0c2bcd2-3ca2-431d-b015-640a3836b410 req-f18e3819-f8ee-49f1-b8f2-1ec342025ee8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:13.136466 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-b0c2bcd2-3ca2-431d-b015-640a3836b410 req-f18e3819-f8ee-49f1-b8f2-1ec342025ee8 service nova] Creating event network-vif-unplugged:af5eca8e-a823-4591-9077-2a4b9b7fb155 for instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 on hv-cinder-maste Sep 18 01:00:13.154441 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-b0c2bcd2-3ca2-431d-b015-640a3836b410 req-f18e3819-f8ee-49f1-b8f2-1ec342025ee8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.103732 Sep 18 01:00:13.156094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 172/338] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:13.658522 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:13.659275 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:13.671739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:13.677004 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:13.681024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:13.685144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:13.688429 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:13.689741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:14.176500 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f638238d-83d3-4ad4-aad6-a3484fcbabf4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Action: 'create', calling method: .version_select at 0x7fcab1db8040>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-58756093", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK/a2QpP2V3S7w6w6BOkpezIW9YzF1VsM9u9w9w2iprgd06lCHf/c7poSqc+Jxfh1V6t4VBzwxsX1Yb5D2hI8qFwBlIbV3JD9H2wQP9ywqVRH/3EBfC48UeaPM9VOhS6yw=="}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:14.205170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-f638238d-83d3-4ad4-aad6-a3484fcbabf4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for project 6958209222b44905bcb4baa1b8c26c58. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:00:14.219143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-f638238d-83d3-4ad4-aad6-a3484fcbabf4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for user 823fa0c3eeeb489591c639b81afc02bc and project 6958209222b44905bcb4baa1b8c26c58. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:00:14.250341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f638238d-83d3-4ad4-aad6-a3484fcbabf4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 398 microversion: 2.1 time: 0.520805 Sep 18 01:00:14.255384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 173/339] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 01:00:13 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:14.269945 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:14.281304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:14.288116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:14.292132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c650fa04-5d9d-4826-8a44-fdfb2c732536 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1722 microversion: 2.20 time: 0.638707 Sep 18 01:00:14.294609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 167/340] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:13 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1722 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:14.885230 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-51dcdbaa-1fe6-4a7e-94d3-3d54bea9b592 req-ede03b81-0a32-4dd4-810a-d18bc0998aef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1a2de33a-66be-4df0-9c70-adf5c1b03ec8", "tag": "b8d36f12-54c1-44a9-97c9-bfd9768a810e"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:14.921615 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-51dcdbaa-1fe6-4a7e-94d3-3d54bea9b592 req-ede03b81-0a32-4dd4-810a-d18bc0998aef service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:14.923061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-51dcdbaa-1fe6-4a7e-94d3-3d54bea9b592 req-ede03b81-0a32-4dd4-810a-d18bc0998aef service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:14.923945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-51dcdbaa-1fe6-4a7e-94d3-3d54bea9b592 req-ede03b81-0a32-4dd4-810a-d18bc0998aef service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:14.952785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-51dcdbaa-1fe6-4a7e-94d3-3d54bea9b592 req-ede03b81-0a32-4dd4-810a-d18bc0998aef service nova] Creating event network-changed:b8d36f12-54c1-44a9-97c9-bfd9768a810e for instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 on hv-cinder-maste Sep 18 01:00:14.975564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-51dcdbaa-1fe6-4a7e-94d3-3d54bea9b592 req-ede03b81-0a32-4dd4-810a-d18bc0998aef service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095319 Sep 18 01:00:14.980711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 174/341] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:00:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:15.320318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:15.321947 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:15.376769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:15.381133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:15.381834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:15.382760 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:15.383371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:15.384972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:15.891284 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:15.904064 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:15.907264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:15.909525 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-92172c12-f8d5-4de0-ae89-edd7187fbd9e tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1725 microversion: 2.20 time: 0.595562 Sep 18 01:00:15.910318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 168/342] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:15 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1725 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:15.939895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e7bc6d89-71b1-435d-815c-bfe81a6b925d"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:15.947061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:15.961443 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:15.963106 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:15.963955 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:15.965724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:15.966873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:15.967552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:16.758855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-96791125-b313-4330-bdfb-a7754a43def7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.826622 Sep 18 01:00:16.761546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 175/343] 10.222.0.42 () {62 vars in 1364 bytes} [Sun Sep 18 01:00:15 2022] POST /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments => generated 188 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 01:00:16.993490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:16.994830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:17.001852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:17.002541 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:17.003157 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:17.004025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:17.004633 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:17.005216 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:17.066418 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120831) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 18 01:00:17.211230 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5058c492-f58c-4e5b-b338-f2c0adfa91e1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/action" status: 202 len: 0 microversion: 2.20 time: 0.222169 Sep 18 01:00:17.212492 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 169/344] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 18 01:00:16 2022] POST /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/action => generated 0 bytes in 224 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:00:17.221432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:17.222054 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:17.251935 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:17.252806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:17.254429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:17.257926 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:17.258643 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:17.262551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:17.685041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:17.699442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:17.702126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:17.703629 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-928befc1-9965-4688-89ee-5d9af2aed636 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1751 microversion: 2.20 time: 0.487221 Sep 18 01:00:17.717093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 176/345] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:17 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1751 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:18.745713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:18.747049 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:18.757966 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:18.758769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:18.760245 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:18.762969 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:18.763607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:18.764196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:19.018213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:19.030062 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:19.031540 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:19.032774 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-928cd202-455a-4006-ae86-1408a2a576ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1751 microversion: 2.20 time: 0.291004 Sep 18 01:00:19.033374 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 170/346] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:18 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1751 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:20.070687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:20.072095 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:20.090359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:20.092843 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:20.093649 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:20.096594 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:20.097076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:20.097530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:20.402831 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:20.416107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:20.418977 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:20.422707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ed964d42-1d70-4abb-99e7-6fb56a73db13 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1753 microversion: 2.20 time: 0.356578 Sep 18 01:00:20.424105 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 177/347] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:20 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1753 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:21.455758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:21.457119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:21.463878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:21.464562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:21.465159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:21.466119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:21.466745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:21.467311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:21.704816 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:21.715013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:21.716661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:21.717896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ecc558ca-d0ec-4565-8c9f-e3774ba36635 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.270266 Sep 18 01:00:21.718447 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 171/348] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:21 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:22.753118 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:22.754699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:22.761772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:22.762470 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:22.763161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:22.764186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:22.764899 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:22.765328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:22.781524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:22.933892 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:22.943923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:22.945820 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:22.947203 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bfd850be-10b9-4416-989c-9cf4601aa779 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.198510 Sep 18 01:00:22.962482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 178/349] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:22 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:23.974511 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:23.975187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:23.982067 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:23.982582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:23.983234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:23.983936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:23.984387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:23.984909 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:24.154426 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:24.164131 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:24.165662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:24.166907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1430ca39-738b-4a32-8ec1-e7bd8876d4b4 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.196503 Sep 18 01:00:24.167458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 172/350] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:23 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:24.713808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6fab853e-b8cd-43f6-9e15-b2bd4c3ff568 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Action: 'create', calling method: .version_select at 0x7fcab1efa700>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-108321933", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAU1QfHgulya8IlMMHM0AYbAmklm4W0dNZ8DZZm6TQfUZA+fydEUfjXTuT+Dysu/N7y1NQULk+7swyt4BTvj5X1+VCmCtqvjkCColq5QocO9UMtsg25vX2ABxOuGmYS3Fg=="}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:24.721548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=4 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:24.731668 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-6fab853e-b8cd-43f6-9e15-b2bd4c3ff568 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Getting quotas for project c9f737f6350f4d1f993708b1f90a862e. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:00:24.735760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-6fab853e-b8cd-43f6-9e15-b2bd4c3ff568 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Getting quotas for user 20a50c6db67f4d818a6fbeaafce299d9 and project c9f737f6350f4d1f993708b1f90a862e. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:00:24.761210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6fab853e-b8cd-43f6-9e15-b2bd4c3ff568 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.054844 Sep 18 01:00:24.761834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 179/351] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 01:00:24 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:00:24.814950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-468345740", "imageRef": "151f4b89-9713-4a41-8534-0e67817b45a2", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-108321933", "networks": [{"uuid": "6fef5f4e-b094-42f4-8864-339fd1327a47"}]}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:24.883463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] validate_networks() for [('6fef5f4e-b094-42f4-8864-339fd1327a47', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:00:25.193884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:25.194829 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:25.226017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:25.226538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:25.226979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:25.239092 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:25.239634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:25.240083 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:25.438074 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:00:25.439383 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:00:25.440174 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:00:25.441498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:00:25.442282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:00:25.462025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:25.462955 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:25.463724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:25.465033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:25.465914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:25.466626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:25.490321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:25.496761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Getting quotas for project c9f737f6350f4d1f993708b1f90a862e. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:00:25.501682 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Getting quotas for user 20a50c6db67f4d818a6fbeaafce299d9 and project c9f737f6350f4d1f993708b1f90a862e. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:00:25.503248 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:25.504915 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:25.505916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3ce92621-a28a-412b-83d4-452b47abd57f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.317013 Sep 18 01:00:25.507450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 180/352] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:25 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:25.519091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:00:25.526226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='151f4b89-9713-4a41-8534-0e67817b45a2',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:00:25.589400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cff5bc88-0590-4710-bfbc-2f2819dfeaac tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.778212 Sep 18 01:00:25.590503 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 173/353] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:00:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 780 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:00:25.601875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9c4a4455-96f4-442e-b125-9139dadf5d69 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:25.602531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9c4a4455-96f4-442e-b125-9139dadf5d69 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:25.670072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9c4a4455-96f4-442e-b125-9139dadf5d69 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:25.677257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9c4a4455-96f4-442e-b125-9139dadf5d69 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1257 microversion: 2.1 time: 0.079875 Sep 18 01:00:25.677839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 181/354] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:25 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1257 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:26.536041 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:26.537355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:26.544071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:26.544595 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:26.545032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:26.545781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:26.546226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:26.546732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:26.691650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:26.701005 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:26.702593 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:26.704124 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:26.707249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 01:00:26.707416 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Sun Sep 18 01:00:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe (ip 10.222.0.42) !!! Sep 18 01:00:26.713446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:26.714732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e92515a5-da80-4f89-af1b-863d9de923b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.182786 Sep 18 01:00:26.715522 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 174/355] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:26 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:26.722202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:26.722988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:26.723634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:26.724545 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:26.725782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:26.726406 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:26.798429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:26.806378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1023db54-0cfd-4f89-8c8f-2c4a5efb7594 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1261 microversion: 2.1 time: 0.109448 Sep 18 01:00:26.806985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 182/356] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:26 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1261 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:27.741672 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:27.742381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:27.748565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:27.749145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:27.749644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:27.750421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:27.750902 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:27.751323 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:27.829243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:27.830093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:27.838997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:27.839988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:27.841445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:27.842714 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:27.843714 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:27.844824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:27.926368 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:27.936003 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-354ea8f6-f65e-48f6-92c7-cafa33d11b12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1261 microversion: 2.1 time: 0.110812 Sep 18 01:00:27.937316 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 183/357] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:27 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1261 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:27.952248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:27.963541 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:27.965129 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:27.966666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b55b2945-325a-469e-8d15-65ac96de47fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.229172 Sep 18 01:00:27.967609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 175/358] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:27 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:28.959083 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:28.960202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:28.966886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:28.967375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:28.967815 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:28.968524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:28.968981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:28.969405 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:28.992091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:28.993427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:29.001461 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:29.002104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:29.002569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:29.003474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:29.003928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:29.004346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:29.037169 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:29.069898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:29.078472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-10f713db-3bb5-4ddb-9d3c-ec0416a4bb31 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1261 microversion: 2.1 time: 0.123230 Sep 18 01:00:29.095695 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 184/359] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:28 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1261 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:29.192829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:29.201887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:29.203369 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:29.204574 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c7dd998c-77cf-4e34-a570-b9877a9518ac tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.217017 Sep 18 01:00:29.205369 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 176/360] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:28 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:30.108113 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:30.109533 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:30.117057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:30.117757 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:30.122498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:30.122498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:30.122498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:30.122498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:30.183486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:30.197941 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-02720677-531d-40b9-b17a-2327af29b214 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1261 microversion: 2.1 time: 0.094021 Sep 18 01:00:30.198551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 185/361] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:30 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1261 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:30.232218 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:30.233510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:30.239977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:30.240647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:30.242058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:30.242978 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:30.243578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:30.244149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:30.392082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:30.401817 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:30.403276 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:30.404798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1f7034dc-5d4b-4597-b607-17a4223eaf2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.177082 Sep 18 01:00:30.405560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 177/362] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:30 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:30.495770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "aecbfe01-aa3f-4ef9-b051-e26e7c98ea22"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:30.498226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:30.504384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:30.504876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:30.505321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:30.506036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:30.506548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:30.506979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:31.221848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:31.222935 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:31.229557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:31.230325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:31.230788 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:31.231495 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:31.233069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:31.233069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:31.299014 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:31.307561 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-124bf1cd-997b-42e1-a004-f162fc243804 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1261 microversion: 2.1 time: 0.089729 Sep 18 01:00:31.308953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 178/363] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:31 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1261 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:31.433017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:31.433623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:31.439728 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:31.440212 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:31.440647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:31.441344 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:31.441802 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:31.442230 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:31.579922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:31.588687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:31.590120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:31.591304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-46fd4b47-d126-4d8b-9523-f58fd77eeb28 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.162781 Sep 18 01:00:31.592239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 179/364] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:31 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:32.331163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:32.331852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:32.338372 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:32.338891 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:32.339325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:32.340014 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:32.340464 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:32.340886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:32.402318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:32.410369 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-997546b3-d46c-43ee-881e-10a359da234c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1261 microversion: 2.1 time: 0.082906 Sep 18 01:00:32.410905 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 180/365] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:32 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1261 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:32.618254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:32.619239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:32.625490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:32.625970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:32.626416 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:32.627125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:32.627573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:32.628043 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:32.764970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:32.775634 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:32.777461 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:32.778729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-687d7304-33fc-4a2f-aded-0b6fb6ca1358 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.164846 Sep 18 01:00:32.779833 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 181/366] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:32 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:33.419566 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-963913806", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-58756093", "security_groups": [{"name": "tempest-secgroup-smoke-821199059"}], "block_device_mapping_v2": [{"uuid": "59129af3-cd17-48cc-9f9b-4f1d0af3cdf7", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "4ff6b087-81f6-47ef-8679-6af29cd7449d"}]}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:33.557968 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] validate_networks() for [('4ff6b087-81f6-47ef-8679-6af29cd7449d', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:00:33.964266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:00:33.964681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:00:33.966085 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:00:33.966534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:00:33.966931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:00:33.977149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:33.977682 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:33.978194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:33.978968 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:33.979401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:33.979819 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:33.996916 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for project 6958209222b44905bcb4baa1b8c26c58. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:00:34.001244 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for user 823fa0c3eeeb489591c639b81afc02bc and project 6958209222b44905bcb4baa1b8c26c58. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:00:34.059736 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:00:34.061788 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='59129af3-cd17-48cc-9f9b-4f1d0af3cdf7',volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:00:34.189767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f4d3ff65-0b22-4123-ab90-660fa58903e8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.774040 Sep 18 01:00:34.190407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 182/367] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:00:33 2022] POST /compute/v2.1/servers => generated 405 bytes in 775 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 18 01:00:34.202347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:34.202989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:34.209658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:34.210150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:34.210624 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:34.211734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:34.211833 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:34.212172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:34.276121 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:34.284184 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-964c8bfc-e218-4091-88e3-a5f70d684ec5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1261 microversion: 2.1 time: 0.085781 Sep 18 01:00:34.284714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 183/368] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:34 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1261 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:34.289207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:34.289808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:34.295986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:34.296490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:34.296887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:34.297572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:34.298022 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:34.298436 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:34.444724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:34.454295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:34.456288 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:34.457449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2216b88e-b903-4dc3-af87-7b7dcf18e415 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.171841 Sep 18 01:00:34.458002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 184/369] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:34 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:34.463033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a6b654b7-756b-4861-83b9-9fbf51554fe5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:34.463623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a6b654b7-756b-4861-83b9-9fbf51554fe5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:34.531238 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a6b654b7-756b-4861-83b9-9fbf51554fe5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:34.538427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a6b654b7-756b-4861-83b9-9fbf51554fe5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1092 microversion: 2.1 time: 0.079199 Sep 18 01:00:34.539001 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 185/370] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:34 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1092 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:34.617373 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9aa885af-b291-4613-94e9-9b6a8ff25e4e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.125567 Sep 18 01:00:34.618026 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 186/371] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:00:30 2022] POST /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments => generated 194 bytes in 4127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:35.308008 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:35.308711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:35.316185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:35.316942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:35.317445 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:35.318975 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:35.319458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:35.319898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:35.395709 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:35.403400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-39a0713c-140a-4cc8-88c1-615dac718b38 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1309 microversion: 2.1 time: 0.100351 Sep 18 01:00:35.403954 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 186/372] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:35 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1309 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:35.483671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:35.484422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:35.492278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:35.492987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:35.493417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:35.494113 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:35.494601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:35.495025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:35.560752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:35.561379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:35.567625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:35.568162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:35.568626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:35.569831 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:35.570296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:35.570751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:35.648906 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:35.653787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:35.658490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cbddc8d5-7a95-45bb-abf3-1d0d9f24c36d tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1134 microversion: 2.1 time: 0.101502 Sep 18 01:00:35.659312 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 187/373] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:35 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1134 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:35.663426 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:35.664859 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:35.666012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f701e775-a5ac-49d0-a73d-1a8d771e3de5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.186727 Sep 18 01:00:35.666582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 187/374] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:35 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:36.428027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:36.429504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:36.431507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:36.449147 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:36.449959 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:36.450405 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:36.451177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:36.452071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:36.452657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:36.681943 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:36.682671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:36.690995 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:36.693064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:36.695922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:36.697063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:36.697625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:36.698382 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:36.699066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:36.699692 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:36.937270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:36.947865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3411a663-5dd7-4e80-bff2-d028ba7f81fa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.270349 Sep 18 01:00:36.948914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 188/375] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:36 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:36.954565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:36.955356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:36.962248 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:36.963237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:36.964079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:36.965132 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:36.965814 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:36.966505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:37.093653 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:37.104552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6b855e5d-ff8e-43bc-ae5d-ac2777745b7c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.680933 Sep 18 01:00:37.110577 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 188/376] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:36 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:37.205750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:37.217220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:37.218702 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:37.220143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d5b5e25a-e9fb-49f3-a6d3-1796f3f7e883 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.270049 Sep 18 01:00:37.221016 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 189/377] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:36 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:37.975210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:37.976686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:37.985347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:37.986396 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:37.987517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:37.988641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:37.989361 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:37.990063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:38.060255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:38.090374 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c9899458-1a6b-485e-9f70-25ef416d7732 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.117178 Sep 18 01:00:38.091279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 189/378] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:37 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:38.132061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:38.132666 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:38.138670 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:38.139155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:38.139605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:38.140284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:38.140732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:38.141149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:38.245188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:38.245795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:38.251717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:38.252194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:38.252624 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:38.253312 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:38.253765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:38.254181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:38.272873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:38.280679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9b6ead65-c8f0-4fee-8c5b-6cf4e6fbe199 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.152453 Sep 18 01:00:38.281454 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 190/379] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:38 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:38.392880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:38.409742 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:38.411591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:38.413533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-55f86122-446f-4893-91cf-cec8ac8a7062 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.171550 Sep 18 01:00:38.414087 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 190/380] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:38 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:39.114147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:39.114796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:39.121544 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:39.122059 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:39.122626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:39.123347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:39.123811 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:39.124243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:39.204688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:39.214992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b6fe2c34-ea27-4b57-ac2b-d58ee9a7840c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.104510 Sep 18 01:00:39.216162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 191/381] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:39 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:39.304681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:39.305293 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:39.313891 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:39.314385 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:39.314843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:39.315547 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:39.316142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:39.316568 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:39.439504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:39.440107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:39.446927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:39.447424 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:39.447739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:39.447879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:39.448553 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:39.449027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:39.449454 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:39.455789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e103c8b0-955b-4457-9483-fea943190cf1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.155662 Sep 18 01:00:39.456356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 191/382] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:39 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:39.579027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:39.589163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:39.590709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:39.593924 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3b8be511-075f-4cf0-bf71-ea335d5f5ce2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.158136 Sep 18 01:00:39.594946 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 192/383] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:39 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:40.240306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:40.240921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:40.250069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:40.250646 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:40.251088 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:40.251796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:40.252290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:40.252722 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:40.329120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:40.339795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5881742c-6d17-477b-8c8d-db9e3e2f4863 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.105067 Sep 18 01:00:40.340845 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 192/384] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:40 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:40.479899 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:40.480538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:40.487398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:40.487923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:40.488360 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:40.489068 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:40.489515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:40.489965 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:40.624171 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:40.624773 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:40.626391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:40.630753 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:40.631233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:40.631658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:40.632350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:40.632796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:40.633213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:40.634520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-daa28ea5-643b-4cfd-93bd-391cebd845c9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.160126 Sep 18 01:00:40.635091 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 193/385] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:40 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:40.782304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:40.791601 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:40.793087 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:40.794248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5ea6e759-63a0-4332-9842-418a156892c6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.173959 Sep 18 01:00:40.794842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 193/386] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:40 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:41.364356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:41.364972 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:41.372066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:41.373389 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:41.373636 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:41.374349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:41.375035 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:41.375279 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:41.437367 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:41.445051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-69f59937-27df-4c2c-b7c6-f7639c84b9c1 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.084600 Sep 18 01:00:41.445626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 194/387] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:41 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:41.663701 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:41.665507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:41.676533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:41.677336 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:41.678124 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:41.679700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:41.680609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:41.681484 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:41.821042 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:41.821662 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:41.829066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:41.829560 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:41.829988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:41.830728 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:41.831181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:41.831593 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:41.836872 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:41.839233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:41.845844 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e0abf827-21b2-4ae3-90dc-e32caf8ed4ba tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.189593 Sep 18 01:00:41.846418 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 194/388] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:41 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:41.965402 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:41.967685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:41.977698 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:41.979209 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:41.980402 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fc0f52bf-3259-4add-a22e-20a2d28f466c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.163901 Sep 18 01:00:41.981301 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 195/389] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:41 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:42.478666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:42.479722 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:42.486420 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:42.487186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:42.487616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:42.488315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:42.488921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:42.489341 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:42.566086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:42.574734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f138968f-bd21-4736-83dc-2c62fa62a122 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.108244 Sep 18 01:00:42.575299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 195/390] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:42 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:42.868539 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:42.869131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:42.876772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:42.877254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:42.877688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:42.878378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:42.878930 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:42.879362 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:43.012316 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:43.013754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:43.020301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:43.021372 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:43.021904 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:43.023049 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:43.023802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:43.024128 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:43.024624 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:43.035339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-96e89655-2615-4629-bd0f-e94be0b2c008 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.170489 Sep 18 01:00:43.038845 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 196/391] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:42 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:43.160045 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:43.168739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:43.170181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:43.171356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2ea6117b-ce60-430c-95fe-20572409c9e6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.164771 Sep 18 01:00:43.171912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 196/392] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:43 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:43.598646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:43.599283 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:43.605840 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:43.606348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:43.606839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:43.607572 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:43.608047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:43.608497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:43.672546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:43.680964 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fbaded5d-d510-4d86-a70a-db579f5a7a51 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.086173 Sep 18 01:00:43.681456 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 197/393] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:43 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:44.063951 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:44.064618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:44.066406 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=4 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:44.072410 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:44.072892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:44.073319 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:44.074004 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:44.074448 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:44.074886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:44.197495 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:44.198106 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:44.201324 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:44.205074 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:44.205548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:44.205981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:44.206692 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:44.207141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:44.207564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:44.209511 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ea895234-6526-473f-95e3-c5187fe8adad tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.149519 Sep 18 01:00:44.210115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 197/394] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:44 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:44.360622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:44.372834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:44.374272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:44.375478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bd55312e-61e4-4197-a2cd-074e40923827 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.182625 Sep 18 01:00:44.376039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 198/395] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:44 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:44.704648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:44.705368 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:44.708691 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=14 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:44.712937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:44.713432 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:44.713864 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:44.714598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:44.715055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:44.715472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:44.808149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:44.815898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a56c1715-202e-416b-a2e4-cf54c6a37dd4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.115293 Sep 18 01:00:44.816379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 198/396] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:44 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:45.239540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:45.240436 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:45.247660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:45.248148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:45.248582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:45.249286 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:45.249738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:45.250162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:45.401323 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:45.403897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:45.405237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:45.410440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e0a9f3b1-fa42-4dc0-a629-3c4b5d99366f tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.180327 Sep 18 01:00:45.410659 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 199/397] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:45 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:45.417193 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:45.418518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:45.419877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:45.422335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:45.423035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:45.425372 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:45.590943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:45.607661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:45.609445 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:45.610790 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3596e9c9-6f41-4047-858c-49c0ffdc6310 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.214140 Sep 18 01:00:45.617557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 199/398] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:45 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:45.840668 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:45.841342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:45.847650 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:45.848120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:45.848552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:45.849824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:45.850338 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:45.850813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:45.919269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:45.927787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-607476bc-0eb0-46e5-9ec5-a5364f3ebb8e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.092099 Sep 18 01:00:45.928579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 200/399] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:45 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:46.435007 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:46.435620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:46.441479 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:46.441951 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:46.442390 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:46.443221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:46.443886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:46.444104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:46.596889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:46.605282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f7efa384-f54b-4113-85d9-b8ee89dcd8cb tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.174839 Sep 18 01:00:46.605837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 200/400] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:46 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:46.636354 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:46.636964 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:46.643292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:46.643764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:46.644195 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:46.644871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:46.645318 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:46.645737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:46.784033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:46.794010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:46.795511 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:46.796786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-abca1395-998b-41e2-8b02-b642a94d1b72 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.164352 Sep 18 01:00:46.797560 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 201/401] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:46 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:46.951825 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:46.953086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:46.959295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:46.959776 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:46.960208 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:46.961387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:46.961834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:46.962252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:47.027922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:47.035929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9a4124a0-0858-484e-a626-6dba0cdf8f91 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.087977 Sep 18 01:00:47.036504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 201/402] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:46 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:47.628551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:47.629155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:47.635887 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:47.636366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:47.636826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:47.637998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:47.637998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:47.638352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:47.773050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:47.781043 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3f42dce6-b5a6-450d-824b-6c79335e9a30 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.156360 Sep 18 01:00:47.781527 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 202/403] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:47 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:47.822912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:47.823739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:47.837793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:47.838386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:47.838933 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:47.839746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:47.840224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:47.840724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:47.971954 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:47.981168 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:47.982632 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:47.983789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5631bc69-059a-4566-a55d-b9663e84b37b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.165957 Sep 18 01:00:47.984335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 202/404] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:47 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:48.060858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:48.061465 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:48.067411 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:48.067892 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:48.068331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:48.069027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:48.069484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:48.070415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:48.133237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:48.140846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b9821537-f1cd-4a80-83ac-148bedc80436 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.083872 Sep 18 01:00:48.141404 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 203/405] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:48 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:48.803562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:48.804167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:48.810832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:48.811310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:48.811765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:48.812455 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:48.812906 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:48.813326 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:48.939657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:48.947257 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7640a07d-aee9-4ef9-903e-bb8cce98b936 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.147588 Sep 18 01:00:48.947820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 203/406] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:48 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:49.009274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:49.009881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:49.016236 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:49.016715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:49.017152 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:49.017863 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:49.018295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:49.018747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:49.151391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:49.160069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:49.161639 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:49.162830 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-10e05b4a-9040-484b-9696-5510a8bb6550 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.157387 Sep 18 01:00:49.163376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 204/407] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:49 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:49.163489 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:49.164592 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:49.170512 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:49.170997 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:49.171421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:49.172107 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:49.172552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:49.172971 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:49.235518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:49.243283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3796ea17-2544-42b7-8688-9ac2dd943ca9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.083764 Sep 18 01:00:49.243837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 204/408] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:49 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:49.969295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:49.969927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:49.975965 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:49.977017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:49.977467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:49.978140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:49.978630 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:49.981911 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:50.188491 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:50.189139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:50.196253 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:50.196745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:50.197210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:50.197917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:50.198498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:50.198992 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:50.445888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:50.454222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4d88422a-86ad-4d86-b560-3f8726503a56 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.488749 Sep 18 01:00:50.466486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 205/409] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:49 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:50.470573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:50.471191 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:50.478141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:50.478684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:50.479132 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:50.479832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:50.480292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:50.480903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:50.488446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:50.497894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:50.499398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:50.500553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4ff91a6a-e7ef-4dfb-b5ad-960d64553b7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.316157 Sep 18 01:00:50.501178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 205/410] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:50 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:50.578180 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:50.586134 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5682b66f-b39a-41c0-aab6-e95cc8d56eb9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1190 microversion: 2.1 time: 0.130016 Sep 18 01:00:50.587110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 206/411] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:50 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1190 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:50.775942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-2376246b-41df-413c-aea4-e93f0db6f65a req-60dff4d5-6b45-4091-8e8d-c91bb907ae55 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35d3affc-6943-43c5-9921-109f6fe429fe", "tag": "d02eceef-bf2a-4aaa-b246-5fafd4b75314"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:50.785359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-2376246b-41df-413c-aea4-e93f0db6f65a req-60dff4d5-6b45-4091-8e8d-c91bb907ae55 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:50.785831 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-2376246b-41df-413c-aea4-e93f0db6f65a req-60dff4d5-6b45-4091-8e8d-c91bb907ae55 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:50.786257 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-2376246b-41df-413c-aea4-e93f0db6f65a req-60dff4d5-6b45-4091-8e8d-c91bb907ae55 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:50.798215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-2376246b-41df-413c-aea4-e93f0db6f65a req-60dff4d5-6b45-4091-8e8d-c91bb907ae55 service nova] Creating event network-changed:d02eceef-bf2a-4aaa-b246-5fafd4b75314 for instance 35d3affc-6943-43c5-9921-109f6fe429fe on hv-cinder-maste Sep 18 01:00:50.803415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-2376246b-41df-413c-aea4-e93f0db6f65a req-60dff4d5-6b45-4091-8e8d-c91bb907ae55 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031668 Sep 18 01:00:50.804002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 206/412] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:00:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:51.477380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:51.477929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:51.484472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:51.484962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:51.485439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:51.486151 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:51.486642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:51.487067 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:51.527234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:51.527876 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:51.534189 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:51.534718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:51.535157 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:51.535864 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:51.536360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:51.536767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:51.728482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:51.754207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fa89c31a-c94d-48e1-9e20-a78beb072a3e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.280558 Sep 18 01:00:51.772385 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 207/413] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:51 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:51.774111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:51.776881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:51.790855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:51.791356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:51.791847 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:51.792502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:51.792957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:51.793960 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:51.831742 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:51.844675 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:51.847425 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:51.848645 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0a9b5139-e812-4405-bd49-73537ebc9992 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.325800 Sep 18 01:00:51.866151 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 207/414] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:51 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:51.930903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:51.939028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d75dd3ed-8b16-4f7b-b36b-9ee634776a4c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1208 microversion: 2.1 time: 0.177758 Sep 18 01:00:51.939671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 208/415] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:51 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1208 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:51.993375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:51.996657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:52.002597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.003083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.003517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:52.004216 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.004691 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.005115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:52.029294 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lazy-loading 'flavor' on Instance uuid 3e1c6d6a-9178-4c19-a032-ef79295fce8c {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:00:52.174271 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a66c1992-9ba5-45a7-9b1e-b867ebb891fa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/action" status: 200 len: 32 microversion: 2.1 time: 0.184948 Sep 18 01:00:52.175179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 208/416] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:00:51 2022] POST /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/action => generated 32 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:00:52.185162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:52.185785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:52.201876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.202395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.203039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:52.203755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.204215 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.204638 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:52.511339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:52.520188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-381d0493-4fbe-4065-be6f-55e0f8a86d6e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.338788 Sep 18 01:00:52.521107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 209/417] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:52 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:52.790477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:52.791918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:52.799374 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.800581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.801248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:52.802185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.802942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.803599 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:52.877029 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:52.878082 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:52.885808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.887349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.888035 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:52.889011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:52.889407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:52.889906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:53.135416 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:53.144528 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e28c610f-e19b-415e-a348-255ba6310d25 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.358957 Sep 18 01:00:53.145712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 209/418] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:52 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:53.152952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:53.153907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:53.160363 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:53.161196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:53.161846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:53.162793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:53.163467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:53.164701 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:53.167832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:53.185139 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:53.199134 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:53.201634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:53.203148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0b1a958d-b7a5-43b7-a5fd-9f81cdb0f8b8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.330941 Sep 18 01:00:53.203688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 210/419] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:52 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:53.209341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-3e984740-6d4b-4dff-b0e1-b57cd4d21928 req-82f729bd-f86b-45cd-bf50-427500384df8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5efa25bd-04ff-49f6-9fd5-6e8c82bcd152", "name": "network-vif-plugged", "status": "completed", "tag": "af5eca8e-a823-4591-9077-2a4b9b7fb155"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:53.220872 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-3e984740-6d4b-4dff-b0e1-b57cd4d21928 req-82f729bd-f86b-45cd-bf50-427500384df8 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:53.221954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-3e984740-6d4b-4dff-b0e1-b57cd4d21928 req-82f729bd-f86b-45cd-bf50-427500384df8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:53.222094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-3e984740-6d4b-4dff-b0e1-b57cd4d21928 req-82f729bd-f86b-45cd-bf50-427500384df8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:53.234712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-3e984740-6d4b-4dff-b0e1-b57cd4d21928 req-82f729bd-f86b-45cd-bf50-427500384df8 service nova] Creating event network-vif-plugged:af5eca8e-a823-4591-9077-2a4b9b7fb155 for instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 on hv-cinder-maste Sep 18 01:00:53.239815 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-3e984740-6d4b-4dff-b0e1-b57cd4d21928 req-82f729bd-f86b-45cd-bf50-427500384df8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034461 Sep 18 01:00:53.240416 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 211/420] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:00:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:53.363915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:53.375462 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-19cf50f1-6b2c-46e9-9a58-e596eb78df87 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.228313 Sep 18 01:00:53.376228 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 210/421] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:53 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:53.548949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:53.549654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:53.556446 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:53.556948 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:53.557359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:53.558171 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:53.558529 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:53.558949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:53.691307 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:53.701260 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-946d2ade-7c24-4e5c-9ee2-aea4f1f83ff6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.157191 Sep 18 01:00:53.701855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 212/422] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:53 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:54.171827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:54.173027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:54.185735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.186228 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.186950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.187740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.188396 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.188814 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.229894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:54.230540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:54.236744 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.237233 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.237675 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.238384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.238950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.239384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.244158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:00:54.495293 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:54.504870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a6fc044a-6694-4b0f-b837-029fdde12032 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.336697 Sep 18 01:00:54.505917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 211/423] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:54 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:54.511721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:54.512692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:54.521783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.522692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.523850 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.524795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.525470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.526110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.571055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:54.591206 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:54.592882 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:54.594250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ad32f81f-58da-4f97-a68f-fb806f814a9f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.368850 Sep 18 01:00:54.594850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 213/424] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:54 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:00:54.728336 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:54.729285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:54.736725 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.738085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.738796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.739529 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:54.740006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:54.740434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:54.837102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:54.849535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-eeebe6cc-dfa9-4dab-a338-1fab21b61620 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.342378 Sep 18 01:00:54.851034 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 212/425] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:54 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:54.961898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:54.970543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-70c84f8a-1fae-47b1-9fa1-7aa5718aa25e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.247882 Sep 18 01:00:54.971314 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 214/426] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:54 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:55.192812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-dec5aff6-f870-4449-87dd-c9a7938de20e req-e2cbbc4f-7168-4067-8a8f-454b25724695 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "443c0c33-d16f-4b9b-86dc-99207ed72962", "tag": "644cd5b5-288e-4651-9d36-20ed8d528fcb"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:55.204029 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-dec5aff6-f870-4449-87dd-c9a7938de20e req-e2cbbc4f-7168-4067-8a8f-454b25724695 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:55.204597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-dec5aff6-f870-4449-87dd-c9a7938de20e req-e2cbbc4f-7168-4067-8a8f-454b25724695 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:55.205091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-dec5aff6-f870-4449-87dd-c9a7938de20e req-e2cbbc4f-7168-4067-8a8f-454b25724695 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:55.219153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-dec5aff6-f870-4449-87dd-c9a7938de20e req-e2cbbc4f-7168-4067-8a8f-454b25724695 service nova] Creating event network-changed:644cd5b5-288e-4651-9d36-20ed8d528fcb for instance 443c0c33-d16f-4b9b-86dc-99207ed72962 on hv-cinder-maste Sep 18 01:00:55.224097 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-dec5aff6-f870-4449-87dd-c9a7938de20e req-e2cbbc4f-7168-4067-8a8f-454b25724695 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034888 Sep 18 01:00:55.224922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 213/427] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:00:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:00:55.528757 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:55.529365 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:55.535332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:55.535811 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:55.536243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:55.536923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:55.537399 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:55.537825 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:55.621946 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:55.622664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:55.629357 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:55.629740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:55.630172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:55.630892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:55.631337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:55.631817 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:55.675044 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:55.682886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-55ced602-2af0-4e75-85b1-763d6d7eca68 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.158125 Sep 18 01:00:55.683556 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 215/428] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:55 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:55.874551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:55.875015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:55.881252 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:55.881654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:55.882337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:55.883477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:55.883768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:55.884177 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:56.088930 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:56.098837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:56.100575 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:56.101768 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ec38d571-04c4-4554-9fd1-55e6c5c2a128 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1809 microversion: 2.20 time: 0.484189 Sep 18 01:00:56.102320 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 214/429] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:55 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1809 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:56.106719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:56.107509 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:56.113325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:56.113798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:56.114240 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:56.115060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:56.115460 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:56.115891 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:56.130718 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:56.139325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e83d12b2-d2dd-4e1d-98bf-d26d2b4c107b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.268669 Sep 18 01:00:56.139833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 216/430] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:55 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:56.329651 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:56.338806 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6f932ead-0b1d-43e2-9678-c88cfb9350dc tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.235461 Sep 18 01:00:56.339378 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 215/431] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:56 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:56.706541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:56.707254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:56.714573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:56.715146 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:56.715644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:56.716411 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:56.716865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:56.717688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:56.857607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:56.866002 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6a8ffb6e-fa00-4b37-bfdc-aa9f3b583153 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.163595 Sep 18 01:00:56.866628 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 217/432] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:56 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:57.128490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:57.129225 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:57.136400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.141665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-e34189f9-7c17-4976-a831-a457b3df73fa req-6ab96e66-f343-4d45-912d-5f0bc06448e0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35d3affc-6943-43c5-9921-109f6fe429fe", "name": "network-vif-plugged", "status": "completed", "tag": "d02eceef-bf2a-4aaa-b246-5fafd4b75314"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:57.147699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.150472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.151926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.152409 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.155291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.161819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-e34189f9-7c17-4976-a831-a457b3df73fa req-6ab96e66-f343-4d45-912d-5f0bc06448e0 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.162430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-e34189f9-7c17-4976-a831-a457b3df73fa req-6ab96e66-f343-4d45-912d-5f0bc06448e0 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.162880 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-e34189f9-7c17-4976-a831-a457b3df73fa req-6ab96e66-f343-4d45-912d-5f0bc06448e0 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.185222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-e34189f9-7c17-4976-a831-a457b3df73fa req-6ab96e66-f343-4d45-912d-5f0bc06448e0 service nova] Creating event network-vif-plugged:d02eceef-bf2a-4aaa-b246-5fafd4b75314 for instance 35d3affc-6943-43c5-9921-109f6fe429fe on hv-cinder-maste Sep 18 01:00:57.190802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-e34189f9-7c17-4976-a831-a457b3df73fa req-6ab96e66-f343-4d45-912d-5f0bc06448e0 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052933 Sep 18 01:00:57.192142 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 218/433] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:00:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:00:57.198435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:57.199324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:57.206993 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.207491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.207923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.208632 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.209092 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.209721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.483226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:57.493890 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:57.495617 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:57.496819 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5d5c4cae-0704-478a-b57d-d92c3d8e03ed tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1796 microversion: 2.20 time: 0.372730 Sep 18 01:00:57.497386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 216/434] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:00:57 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1796 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:00:57.501952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:57.502566 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:57.508931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.509331 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.510536 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.511072 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.511755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.512345 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.537482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:57.546441 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d14f5777-c1e9-4b9e-9dfa-26c68a5a1242 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.352852 Sep 18 01:00:57.547142 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 219/435] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:57 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:57.663696 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:57.671954 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-58f7df53-bd61-4861-8014-82cac1da1b8e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.173573 Sep 18 01:00:57.672523 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 217/436] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:57 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:57.889216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:57.889833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:57.896617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.897109 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.897547 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:57.898251 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:57.898746 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:57.899176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.043164 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:58.052469 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6549ca14-7a9c-41a3-8413-79757bda83c4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1357 microversion: 2.1 time: 0.167233 Sep 18 01:00:58.053122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 220/437] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:57 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1357 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:58.078550 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:58.079222 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:58.085566 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.086274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.086912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.087639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.088239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.088723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.122891 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1b4b5ffc-b927-467d-82d8-738dc065bc02 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8/ips" status: 200 len: 148 microversion: 2.1 time: 0.048041 Sep 18 01:00:58.123259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 218/438] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:00:58 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8/ips => generated 148 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:58.136398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0838177b-6c5f-497e-807d-d10c52962e00"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:00:58.138881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:58.145904 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.146440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.146975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.147730 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.148712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.149197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.246179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Invalid volume: volume 0838177b-6c5f-497e-807d-d10c52962e00 is already attached to instances: 0368fcd1-89d7-4e1f-ad08-1425f3872064 Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: POST /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8/os-volume_attachments HTTP/1.1 Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 74 Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:00:58.249541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {"volumeAttachment": {"volumeId": "0838177b-6c5f-497e-807d-d10c52962e00"}} Sep 18 01:00:58.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Invalid volume: volume 0838177b-6c5f-497e-807d-d10c52962e00 is already attached to instances: 0368fcd1-89d7-4e1f-ad08-1425f3872064 Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'server_id': '1a2de33a-66be-4df0-9c70-adf5c1b03ec8', 'body': {'volumeAttachment': {'volumeId': '0838177b-6c5f-497e-807d-d10c52962e00'}}} Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 0838177b-6c5f-497e-807d-d10c52962e00 is already attached to instances: 0368fcd1-89d7-4e1f-ad08-1425f3872064 Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:00:58.252069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 0838177b-6c5f-497e-807d-d10c52962e00 is already attached to instances: 0368fcd1-89d7-4e1f-ad08-1425f3872064 Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:00:58.253434 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 0838177b-6c5f-497e-807d-d10c52962e00 is already attached to instances: 0368fcd1-89d7-4e1f-ad08-1425f3872064 Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 400 to user: Invalid volume: volume 0838177b-6c5f-497e-807d-d10c52962e00 is already attached to instances: 0368fcd1-89d7-4e1f-ad08-1425f3872064 {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-71e01ec4-a3f6-4998-95b2-033b19060ce1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.120056 Sep 18 01:00:58.254862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 221/439] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:00:58 2022] POST /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8/os-volume_attachments => generated 174 bytes in 122 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:00:58.262267 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:58.262783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:58.268908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.269395 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.269831 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.270666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.271132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.271557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.297471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:00:58.377858 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-75b6d5e6-e89b-450d-b39c-2aa64c548564 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 204 len: 0 microversion: 2.1 time: 0.119587 Sep 18 01:00:58.378374 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 219/440] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:00:58 2022] DELETE /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:00:58.386582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:58.387284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:58.394336 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.394861 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.395355 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.396059 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.396515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.396947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.542837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:58.550835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a180ab66-62e8-4a75-9fc1-bc47203b7b59 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1680 microversion: 2.1 time: 0.168040 Sep 18 01:00:58.551417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 222/441] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:58 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1680 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:58.586928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:58.587628 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:58.594195 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.594730 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.595170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.595884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.596338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.596754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.704442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:58.706098 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:58.714896 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.715392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.715834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.716563 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:58.717022 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:58.718013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:58.760539 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:58.774101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-89ec1a3b-d94a-4d27-9889-49db01450ead tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.190407 Sep 18 01:00:58.774754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 220/442] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:58 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:58.862674 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:58.871814 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-93d33573-55a5-436c-95bf-8420e267d20e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.178074 Sep 18 01:00:58.872397 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 223/443] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:58 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:59.077130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:59.077989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:59.084022 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.084510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.084947 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.085700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.086154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.086870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.223487 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:59.231923 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2c4b4082-dc84-4740-9bcf-67ef5cddc266 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1547 microversion: 2.1 time: 0.158627 Sep 18 01:00:59.232506 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 221/444] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:59 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1547 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:59.254883 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:59.255499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:59.262961 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.263634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.264085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.264795 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.265255 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.265679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.409733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:59.419827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2368bced-be27-4357-8ab8-c3c58207d435 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1547 microversion: 2.1 time: 0.168805 Sep 18 01:00:59.420444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 224/445] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:59 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1547 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:00:59.442852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:59.447847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:59.448619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Searching by: {'deleted': False, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e'} {{(pid=120831) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 18 01:00:59.455724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.456208 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.456639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.457362 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.457815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.458239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.563282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.multi_cell_list [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120831) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 18 01:00:59.574902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:59.575641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:59.582862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.583353 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.583786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.584497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.585106 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.585533 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.669383 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e0579824-6684-4f0e-9cc2-62cc54402121 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.230080 Sep 18 01:00:59.670014 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 222/446] 10.222.0.42 () {58 vars in 1125 bytes} [Sun Sep 18 01:00:59 2022] GET /compute/v2.1/servers => generated 342 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:00:59.681711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:59.682319 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:59.688490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.688987 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.689421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.690131 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.690636 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.691070 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.740398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:59.749013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-067af2b2-6692-48ee-9f00-dc28d5d8fa46 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1680 microversion: 2.1 time: 0.178381 Sep 18 01:00:59.749601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 225/447] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:59 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1680 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:59.797976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:59.798631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:59.806769 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.807567 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.808256 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.809401 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.810142 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.810870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.901250 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:00:59.909568 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-41429539-652c-44ab-a465-f36ae381671a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1547 microversion: 2.1 time: 0.231612 Sep 18 01:00:59.922571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 223/448] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:59 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1547 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:00:59.922571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:00:59.922571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:00:59.923146 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.923352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.923791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:00:59.924504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:00:59.924963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:00:59.925393 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:00.013131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:00.022210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d6db931c-c36a-44b7-8489-47e4ba5276ab tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.228219 Sep 18 01:01:00.022871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 226/449] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:59 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:00.120598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:00.129648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-36eb292d-cffe-4cac-ad9f-973d1217bd63 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.218073 Sep 18 01:01:00.130236 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 224/450] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:00:59 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:00.773352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:00.773964 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:00.780617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:00.781043 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:00.781483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:00.782187 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:00.782708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:00.783138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:00.995061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:01.003140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-03946e21-2d9e-4745-9f95-178020fd5d57 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1680 microversion: 2.1 time: 0.233654 Sep 18 01:01:01.003939 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 227/451] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:00 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1680 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:01.047221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:01.048663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:01.055900 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:01.056394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:01.056834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:01.057562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:01.058025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:01.058445 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:01.154094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:01.154903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:01.161540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:01.162032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:01.162509 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:01.163235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:01.164238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:01.164944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:01.433068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:01.435556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:01.443032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-94b508c4-c42b-4478-a47a-d691a9bfac94 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.399709 Sep 18 01:01:01.453042 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 225/452] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:01 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:01.466577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:01.476960 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5854dcfd-42a6-40fe-a774-079624e3a693 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.326044 Sep 18 01:01:01.477886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 228/453] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:01 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:01.540729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "502c5bc8-cddb-4feb-9f92-837885ed54b7"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:01.543144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:01.549926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:01.550405 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:01.550889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:01.551586 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:01.552083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:01.552516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:01.885277 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5e005731-4424-4b4a-a968-1b22a3d9dbf0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.348795 Sep 18 01:01:01.886562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 226/454] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:01:01 2022] POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments => generated 194 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:01.987927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-0cd0f961-f061-4c66-9db8-91e1d3a5bba2 req-135f0879-ab3b-45c9-954f-17b81a285382 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1a2de33a-66be-4df0-9c70-adf5c1b03ec8", "name": "network-vif-deleted", "tag": "b8d36f12-54c1-44a9-97c9-bfd9768a810e"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:01.999747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0cd0f961-f061-4c66-9db8-91e1d3a5bba2 req-135f0879-ab3b-45c9-954f-17b81a285382 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:02.000628 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0cd0f961-f061-4c66-9db8-91e1d3a5bba2 req-135f0879-ab3b-45c9-954f-17b81a285382 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:02.001297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0cd0f961-f061-4c66-9db8-91e1d3a5bba2 req-135f0879-ab3b-45c9-954f-17b81a285382 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:02.014171 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-0cd0f961-f061-4c66-9db8-91e1d3a5bba2 req-135f0879-ab3b-45c9-954f-17b81a285382 service nova] Creating event network-vif-deleted:b8d36f12-54c1-44a9-97c9-bfd9768a810e for instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 on hv-cinder-maste Sep 18 01:01:02.019616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-0cd0f961-f061-4c66-9db8-91e1d3a5bba2 req-135f0879-ab3b-45c9-954f-17b81a285382 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035253 Sep 18 01:01:02.028407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:02.029414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:02.034021 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 229/455] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:02.039762 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:02.040255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:02.040689 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:02.041475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:02.042129 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:02.042767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:02.113304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:02.121556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-288fbd9e-b3a1-4c18-a266-119e4c9ebbee tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 200 len: 1326 microversion: 2.1 time: 0.097525 Sep 18 01:01:02.122140 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 227/456] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:02 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 1326 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:02.471724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:02.472802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:02.483774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:02.484871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:02.485870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:02.487153 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:02.488116 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:02.489090 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:02.509572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:02.511927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:02.527432 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:02.536100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:02.540887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:02.542640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:02.543880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:02.545144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:02.775848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:02.784561 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a0f8c360-b1d2-4b06-b8f3-9b886143a808 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.282268 Sep 18 01:01:02.785146 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 228/457] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:02 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:02.785654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:02.796666 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-857e3949-4808-4931-8a9f-dbaba9d0883a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.332465 Sep 18 01:01:02.803013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 230/458] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:02 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:03.160396 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:03.161926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:03.173317 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:03.174613 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:03.175456 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:03.176752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:03.177662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:03.179914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:03.201485 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 HTTP/1.1 Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:01:03.204090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:01:03.205493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '1a2de33a-66be-4df0-9c70-adf5c1b03ec8'} Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:01:03.206235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:01:03.207423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:03.211874 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:01:03.213463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0d3c84b0-9c55-4b18-ad2d-5d47c3e1ea74 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 404 len: 111 microversion: 2.1 time: 0.058259 Sep 18 01:01:03.214731 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 229/459] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:03 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:01:03.235947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0df1572e-d7b4-47d0-a0b6-7eebfcc21ec1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '.version_select at 0x7fcab1dc8310>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:03.266789 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0df1572e-d7b4-47d0-a0b6-7eebfcc21ec1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-467033041" status: 202 len: 0 microversion: 2.1 time: 0.047573 Sep 18 01:01:03.267376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 231/460] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 18 01:01:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-467033041 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:01:03.810018 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:03.810808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:03.818558 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:03.819059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:03.819500 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:03.820226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:03.820751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:03.821421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:03.825164 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:03.826984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:03.839745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:03.841167 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:03.842265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:03.843821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:03.845112 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:03.845984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:03.855719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:04.118658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:04.131673 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-071866d5-6afa-404e-ad0d-d0730a46601b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.325533 Sep 18 01:01:04.151494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 230/461] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:03 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:04.198240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:04.209829 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-83888af9-26a6-4e3e-a5ce-50cd3c663f67 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.393048 Sep 18 01:01:04.211642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 232/462] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:03 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:04.461009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:04.461564 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:04.468111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:04.468617 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:04.469005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:04.469702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:04.470692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:04.470692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:04.607700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lazy-loading 'services' on Instance uuid 0368fcd1-89d7-4e1f-ad08-1425f3872064 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:01:04.730308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4b61d372-a4ad-49b0-8084-e086ec2bf0fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments/0838177b-6c5f-497e-807d-d10c52962e00" status: 202 len: 0 microversion: 2.1 time: 0.273045 Sep 18 01:01:04.730922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 231/463] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:01:04 2022] DELETE /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments/0838177b-6c5f-497e-807d-d10c52962e00 => generated 0 bytes in 275 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:01:04.740303 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:04.743245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:04.749687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:04.750179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:04.750656 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:04.751484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:04.751834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:04.752305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:04.784249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b1738419-2ad1-4dd1-addf-27c692c58f5c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048253 Sep 18 01:01:04.784824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 233/464] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:01:04 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:05.158609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:05.159226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:05.166134 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:05.166652 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:05.167088 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:05.167778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:05.168226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:05.168645 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:05.243864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:05.245180 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:05.252249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:05.253064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:05.253790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:05.255762 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:05.256417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:05.257987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:05.346538 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:05.355377 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-67b497ce-55e4-435a-b699-ff6d2244428f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.201185 Sep 18 01:01:05.355958 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 232/465] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:05 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:05.416894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:05.425408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8b5c2745-434e-4a1f-aa90-61f3ead715e2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.187689 Sep 18 01:01:05.426050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 234/466] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:05 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:05.797545 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:05.799713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:05.806611 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:05.807093 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:05.807531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:05.808207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:05.808655 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:05.809076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:05.838983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-60608895-2d0d-4fcc-ba00-1bc0f724e184 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045516 Sep 18 01:01:05.839534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 233/467] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:01:05 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:01:06.384473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:06.385148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:06.392004 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:06.392560 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:06.393006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:06.393707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:06.394161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:06.394629 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:06.452311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:06.453246 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:06.459194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:06.459674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:06.460180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:06.460908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:06.461479 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:06.461855 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:06.567534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:06.576981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6c057fe9-dcdd-44f5-ab18-b2b6bff5fef4 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.196938 Sep 18 01:01:06.580597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 235/468] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:06 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:06.613152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:06.621347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9e7e2a45-b5ac-483f-8986-2df0d038d5cc tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.173090 Sep 18 01:01:06.621903 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 234/469] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:06 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:06.852063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:06.854237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:06.856163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=97,nova_api:DELETE=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:06.861372 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:06.861852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:06.862282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:06.862989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:06.863437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:06.863852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:06.894956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e935133d-3b64-424f-aeab-561345cb33f8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047321 Sep 18 01:01:06.895498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 236/470] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:01:06 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:07.600812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:07.601414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:07.603918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:07.609253 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:07.609737 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:07.610172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:07.610892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:07.611343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:07.611994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:07.644622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:07.645238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:07.652583 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:07.653064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:07.653493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:07.654189 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:07.654671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:07.655096 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:07.796617 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:07.806188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-460e4c38-571b-4b8c-a3ae-6971423fb2c9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.208632 Sep 18 01:01:07.821889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 235/471] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:07 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:07.828605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:07.836941 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d58814d3-9d59-4b7e-8c3d-32517d9fb37e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.196169 Sep 18 01:01:07.837511 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 237/472] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:07 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:07.907401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:07.909644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:07.915452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:07.915953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:07.916365 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:07.917038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:07.917483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:07.917898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:07.947139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-21da87af-f563-4e51-90e0-8469f3114922 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043631 Sep 18 01:01:07.947682 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 236/473] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:01:07 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:08.829277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:08.829883 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:08.836922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:08.837398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:08.837827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:08.838536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:08.838986 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:08.839440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:08.860313 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:08.860928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:08.867468 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:08.867969 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:08.868400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:08.869086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:08.869539 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:08.869961 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:09.048838 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:09.052975 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:09.059706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-047fd7b4-df00-4fb6-b84d-20359e3ed842 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.234483 Sep 18 01:01:09.060286 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 238/474] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:08 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:09.061511 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ee1b30e7-21fd-4161-864b-4632725592ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.204959 Sep 18 01:01:09.064990 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:09.067458 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:09.074905 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:09.075549 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 237/475] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:08 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:09.075751 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:09.076394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:09.077410 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:09.078120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:09.078626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:09.112078 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f727ee1c-fc08-454b-9d2a-358c2bfa1c81 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050553 Sep 18 01:01:09.112712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 239/476] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:01:09 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:09.218675 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-7669a3bb-c5fc-4400-9f55-d82f9fbfbc71 req-16babf95-55ac-46de-bcba-54bf84d662cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "443c0c33-d16f-4b9b-86dc-99207ed72962", "name": "network-vif-plugged", "status": "completed", "tag": "644cd5b5-288e-4651-9d36-20ed8d528fcb"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:09.228632 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-7669a3bb-c5fc-4400-9f55-d82f9fbfbc71 req-16babf95-55ac-46de-bcba-54bf84d662cf service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:09.229237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-7669a3bb-c5fc-4400-9f55-d82f9fbfbc71 req-16babf95-55ac-46de-bcba-54bf84d662cf service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:09.229853 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-7669a3bb-c5fc-4400-9f55-d82f9fbfbc71 req-16babf95-55ac-46de-bcba-54bf84d662cf service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:09.241454 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-7669a3bb-c5fc-4400-9f55-d82f9fbfbc71 req-16babf95-55ac-46de-bcba-54bf84d662cf service nova] Creating event network-vif-plugged:644cd5b5-288e-4651-9d36-20ed8d528fcb for instance 443c0c33-d16f-4b9b-86dc-99207ed72962 on hv-cinder-maste Sep 18 01:01:09.246407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-7669a3bb-c5fc-4400-9f55-d82f9fbfbc71 req-16babf95-55ac-46de-bcba-54bf84d662cf service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031456 Sep 18 01:01:09.247033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 238/477] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:09.649112 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:09.649678 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:09.657350 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:09.658071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:09.658640 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:09.659415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:09.659915 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:09.660345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:09.835120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:09.843772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f833429b-41a0-4929-8c7d-80d5b19dd335 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1593 microversion: 2.1 time: 0.198832 Sep 18 01:01:09.844342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 240/478] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:09 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1593 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:10.087298 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:10.087511 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:10.088007 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:10.088092 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:10.093603 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:10.094534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:10.095582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:10.095981 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:10.096165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:10.096645 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:10.096802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:10.096963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:10.097234 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:10.097783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:10.097912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:10.098242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:10.098705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:10.302790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:10.310287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:10.311859 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-577daa2f-5817-4dcb-9114-6c13c665a085 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1263 microversion: 2.1 time: 0.229351 Sep 18 01:01:10.319560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-28c58109-75f8-423c-a00d-17101f269d8d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.236991 Sep 18 01:01:10.320919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 239/479] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:10 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:10.326003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:10.326905 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 241/480] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:10 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1263 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:10.328631 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:10.335683 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:10.336278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:10.336708 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:10.337409 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:10.337921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:10.338338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:10.383162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-410c3732-e37d-4db4-8993-561cc38d62e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059680 Sep 18 01:01:10.383721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 240/481] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:01:10 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:11.336408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:11.337019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:11.344010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.344783 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.345219 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.345926 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.346382 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.346834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.349716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:11.350604 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:11.358201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.358855 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.359418 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.360398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.361048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.361577 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.562195 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:11.570303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:11.571292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3081570e-8b56-45fc-8bff-73513c84add9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1450 microversion: 2.1 time: 0.238857 Sep 18 01:01:11.572144 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 242/482] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:11 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1450 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:11.576663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:11.580832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-95bccf19-f7d4-4d77-a853-df9f4f5fc55b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.235493 Sep 18 01:01:11.581917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 241/483] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:11 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:11.589912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:11.592741 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:11.599500 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:11.600102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.600241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:11.600589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.601530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.602224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.602735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.603546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.607532 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.608019 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.608446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.609138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.609586 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.610003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.637056 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6564234e-90f6-46e5-85a5-a6370ee660ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052964 Sep 18 01:01:11.637933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 243/484] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:01:11 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064/os-volume_attachments => generated 25 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:01:11.737763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:11.738486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:11.751343 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.751489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.751848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.752544 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.755711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.756508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.794712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:11.795491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:01:11.808242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7d3743cc-d76b-4b8f-b2f8-4fdd0d8bc5d9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1450 microversion: 2.1 time: 0.212059 Sep 18 01:01:11.810785 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 242/485] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:11 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1450 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:11.815607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-63ff3208-83ed-4d94-8e3f-44c43b735ad7 req-0ad9e648-ce53-4863-ae6a-0205bb27ef0b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35d3affc-6943-43c5-9921-109f6fe429fe", "tag": "d02eceef-bf2a-4aaa-b246-5fafd4b75314"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:11.834573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-63ff3208-83ed-4d94-8e3f-44c43b735ad7 req-0ad9e648-ce53-4863-ae6a-0205bb27ef0b service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.835883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-63ff3208-83ed-4d94-8e3f-44c43b735ad7 req-0ad9e648-ce53-4863-ae6a-0205bb27ef0b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.837009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-63ff3208-83ed-4d94-8e3f-44c43b735ad7 req-0ad9e648-ce53-4863-ae6a-0205bb27ef0b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.854758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-63ff3208-83ed-4d94-8e3f-44c43b735ad7 req-0ad9e648-ce53-4863-ae6a-0205bb27ef0b service nova] Creating event network-changed:d02eceef-bf2a-4aaa-b246-5fafd4b75314 for instance 35d3affc-6943-43c5-9921-109f6fe429fe on hv-cinder-maste Sep 18 01:01:11.861656 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-63ff3208-83ed-4d94-8e3f-44c43b735ad7 req-0ad9e648-ce53-4863-ae6a-0205bb27ef0b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049492 Sep 18 01:01:11.868700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 243/486] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:11.933945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8898e5ba-d3a6-42a7-b02f-012729538c04 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 204 len: 0 microversion: 2.1 time: 0.203952 Sep 18 01:01:11.938692 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 244/487] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:01:11 2022] DELETE /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 18 01:01:11.944491 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:11.945276 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:11.957755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.958250 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.958860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:11.959576 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:11.960030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:11.961104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:12.135099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:12.143482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0514ae0c-005e-4d1f-87d4-d87afd38cd0a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1677 microversion: 2.1 time: 0.203123 Sep 18 01:01:12.144650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 244/488] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:11 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1677 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:12.606109 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:12.606917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:12.613925 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:12.614432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:12.614959 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:12.615786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:12.616103 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:12.617058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:12.832819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:12.841962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-05815100-d2d4-432c-84ca-14f58764a4e7 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.240330 Sep 18 01:01:12.843226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 245/489] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:12 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:13.168137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:13.168993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:13.176228 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:13.176966 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:13.177834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:13.179006 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:13.180215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:13.180805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:13.313442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-773960957"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:13.314181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:13.321541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:13.322031 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:13.322519 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:13.323350 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:13.323804 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:13.324228 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:13.396610 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:13.406119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7ded6bc8-a82d-49a0-bd38-14949c375557 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1677 microversion: 2.1 time: 0.242355 Sep 18 01:01:13.406891 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 245/490] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:13 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1677 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:13.486476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.network.security_group_api [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Adding security group fd6ca814-4786-4459-8a91-d5dda7d3826e to port d02eceef-bf2a-4aaa-b246-5fafd4b75314 Sep 18 01:01:13.871748 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:13.872880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:13.882486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:13.883696 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:13.884198 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:13.885002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:13.886222 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:13.886773 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:13.986546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d597884a-25b8-415b-a9a8-595d4210c54e tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/action" status: 202 len: 0 microversion: 2.1 time: 0.676929 Sep 18 01:01:13.987612 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 246/491] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:01:13 2022] POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/action => generated 0 bytes in 679 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:01:13.992706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-9a148f94-0621-42f0-aa84-d88fbe5876b1 req-91d64324-10df-46c2-9269-4744f7a77a89 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "443c0c33-d16f-4b9b-86dc-99207ed72962", "tag": "644cd5b5-288e-4651-9d36-20ed8d528fcb"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:14.002570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-9a148f94-0621-42f0-aa84-d88fbe5876b1 req-91d64324-10df-46c2-9269-4744f7a77a89 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:14.003376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-9a148f94-0621-42f0-aa84-d88fbe5876b1 req-91d64324-10df-46c2-9269-4744f7a77a89 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:14.004062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-9a148f94-0621-42f0-aa84-d88fbe5876b1 req-91d64324-10df-46c2-9269-4744f7a77a89 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:14.017239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-9a148f94-0621-42f0-aa84-d88fbe5876b1 req-91d64324-10df-46c2-9269-4744f7a77a89 service nova] Creating event network-changed:644cd5b5-288e-4651-9d36-20ed8d528fcb for instance 443c0c33-d16f-4b9b-86dc-99207ed72962 on hv-cinder-maste Sep 18 01:01:14.024418 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-9a148f94-0621-42f0-aa84-d88fbe5876b1 req-91d64324-10df-46c2-9269-4744f7a77a89 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035331 Sep 18 01:01:14.025027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 247/492] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:14.036624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:14.037326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:14.044859 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:14.045343 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:14.045771 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:14.046786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:14.047316 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:14.047670 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:14.052205 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:14.203778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:14.213540 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-36a294c7-15a6-4407-8001-456d94ee2c9e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.351285 Sep 18 01:01:14.227349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 246/493] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:13 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:14.236536 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-264b0793-aaaf-4639-ab12-d742990947cf req-e8035139-f65b-41ed-b6e1-4e9ecc8cfb87 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35d3affc-6943-43c5-9921-109f6fe429fe", "tag": "d02eceef-bf2a-4aaa-b246-5fafd4b75314"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:14.247559 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-264b0793-aaaf-4639-ab12-d742990947cf req-e8035139-f65b-41ed-b6e1-4e9ecc8cfb87 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:14.248402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-264b0793-aaaf-4639-ab12-d742990947cf req-e8035139-f65b-41ed-b6e1-4e9ecc8cfb87 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:14.248991 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-264b0793-aaaf-4639-ab12-d742990947cf req-e8035139-f65b-41ed-b6e1-4e9ecc8cfb87 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:14.265221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-264b0793-aaaf-4639-ab12-d742990947cf req-e8035139-f65b-41ed-b6e1-4e9ecc8cfb87 service nova] Creating event network-changed:d02eceef-bf2a-4aaa-b246-5fafd4b75314 for instance 35d3affc-6943-43c5-9921-109f6fe429fe on hv-cinder-maste Sep 18 01:01:14.271091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-264b0793-aaaf-4639-ab12-d742990947cf req-e8035139-f65b-41ed-b6e1-4e9ecc8cfb87 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042190 Sep 18 01:01:14.272245 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 247/494] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:14.340105 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:14.351608 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7d882c4c-b673-426f-a639-44d0ecd13068 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1639 microversion: 2.1 time: 0.325120 Sep 18 01:01:14.367472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 248/495] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:14 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1639 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:14.433323 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:14.434446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:14.440988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:14.441573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:14.442095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:14.443063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:14.443980 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:14.445151 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:14.648667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:14.656979 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-13d69eb5-398e-4bd5-a46d-5621a429c450 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1677 microversion: 2.1 time: 0.227432 Sep 18 01:01:14.670943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 248/496] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:14 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1677 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:15.249375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:15.250947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:15.262713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:15.263617 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:15.264662 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:15.266395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:15.267517 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:15.268889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:15.647380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:15.658872 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1a037bee-aebc-45c1-bce9-c78db2ded2b8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.420363 Sep 18 01:01:15.671260 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 249/497] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:15 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:15.681087 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:15.682621 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:15.689690 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:15.690520 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:15.691815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:15.693028 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:15.693700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:15.694336 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:15.779481 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:15.788849 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f82a793d-fa0c-4392-a2a0-4c671dffcf44 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 200 len: 1610 microversion: 2.1 time: 0.112106 Sep 18 01:01:15.789426 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 249/498] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:15 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 1610 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:16.042966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-4e3e7acd-7742-44fe-84d3-5734ba6ca3e1 req-6bd7b193-4674-4fce-ac40-acc127164d69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0368fcd1-89d7-4e1f-ad08-1425f3872064", "name": "network-vif-deleted", "tag": "190e8866-66ef-440c-9bb4-625459014597"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:16.057962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-4e3e7acd-7742-44fe-84d3-5734ba6ca3e1 req-6bd7b193-4674-4fce-ac40-acc127164d69 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:16.058782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-4e3e7acd-7742-44fe-84d3-5734ba6ca3e1 req-6bd7b193-4674-4fce-ac40-acc127164d69 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:16.059251 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-4e3e7acd-7742-44fe-84d3-5734ba6ca3e1 req-6bd7b193-4674-4fce-ac40-acc127164d69 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:16.072610 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-4e3e7acd-7742-44fe-84d3-5734ba6ca3e1 req-6bd7b193-4674-4fce-ac40-acc127164d69 service nova] Creating event network-vif-deleted:190e8866-66ef-440c-9bb4-625459014597 for instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 on hv-cinder-maste Sep 18 01:01:16.077566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-4e3e7acd-7742-44fe-84d3-5734ba6ca3e1 req-6bd7b193-4674-4fce-ac40-acc127164d69 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038481 Sep 18 01:01:16.078801 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 250/499] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:16.684235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:16.685015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:16.692515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:16.693445 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:16.694062 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:16.694981 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:16.695443 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:16.695870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:16.814799 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:16.815489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:16.822794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:16.823611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:16.824844 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:16.825718 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:16.826445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:16.827171 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:16.845991 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 HTTP/1.1 Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:01:16.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:01:16.849181 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '0368fcd1-89d7-4e1f-ad08-1425f3872064'} Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:01:16.851634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:01:16.853025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:01:16.854994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:01:16.854994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-531c97a8-6676-4d6e-959b-3e289e3829fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 404 len: 111 microversion: 2.1 time: 0.039735 Sep 18 01:01:16.854994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 251/500] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:16 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:01:16.858536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4101e244-5e14-4a1f-9f60-37b0c2d69b14 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '.version_select at 0x7fcab1f25dc0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:16.861278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:16.871413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a93d99cc-844d-4f04-b6d7-eb23eeb6d376 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.192001 Sep 18 01:01:16.872366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 250/501] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:16 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:16.889464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4101e244-5e14-4a1f-9f60-37b0c2d69b14 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-522796901" status: 202 len: 0 microversion: 2.1 time: 0.034529 Sep 18 01:01:16.890044 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 252/502] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 18 01:01:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-522796901 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:01:17.507369 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3902fcfc-8053-4824-9292-d5af80f8824d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: .version_select at 0x7fcab1c39280>, body: {"keypair": {"name": "tempest-keypair-227418299", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCcokLkIC29+dI2i0t0N2e2E+Epn7HHv7rg0CXirI5Tv56BEYX1M1ka8nAPkkW8g1uw0Incq/7kIFOkBmWEWEvuRth1+rrJSbHTWwku4LfkKkzbz9yk18YJqQS3kUxNH1Q=="}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:17.522716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-3902fcfc-8053-4824-9292-d5af80f8824d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:01:17.526532 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-3902fcfc-8053-4824-9292-d5af80f8824d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:01:17.548666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3902fcfc-8053-4824-9292-d5af80f8824d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.045103 Sep 18 01:01:17.549277 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 251/503] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 01:01:17 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:17.898419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:17.899259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:17.906263 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:17.906705 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:17.907140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:17.907840 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:17.908293 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:17.908722 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:18.050335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:18.060876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1e251341-c20e-4eea-a386-b907fb378c60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.167111 Sep 18 01:01:18.061543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 253/504] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:17 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:18.823988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-252546441", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3227bd19-4c4c-45c0-aee5-e73b3870bae1"}], "security_groups": [{"name": "tempest-securitygroup--1466741863"}], "key_name": "tempest-keypair-227418299", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:18.927077 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] validate_networks() for [('3227bd19-4c4c-45c0-aee5-e73b3870bae1', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:01:19.084871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:19.086129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:19.092892 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:19.093376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:19.093819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:19.094609 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:19.095072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:19.095498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:19.328287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:19.337334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bb2208df-0f50-4399-9e81-902f94575968 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.256325 Sep 18 01:01:19.337895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 254/505] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:19 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:19.368583 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:01:19.368999 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:01:19.369466 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:01:19.369865 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:01:19.370273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:01:19.378887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:19.379361 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:19.379786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:19.380501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:19.380941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:19.381356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:19.397211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:01:19.401718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:01:19.412399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:01:19.415370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:01:19.461645 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a9c92c07-4161-4239-b028-258c1d654766 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.642876 Sep 18 01:01:19.462302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 252/506] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:01:18 2022] POST /compute/v2.1/servers => generated 402 bytes in 644 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:01:19.473347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-83ed4b3e-7798-4b01-935b-d75ff5b018d0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:19.474052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-83ed4b3e-7798-4b01-935b-d75ff5b018d0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:19.536210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-83ed4b3e-7798-4b01-935b-d75ff5b018d0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:19.543290 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-83ed4b3e-7798-4b01-935b-d75ff5b018d0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1240 microversion: 2.1 time: 0.073805 Sep 18 01:01:19.543977 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 255/507] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:19 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1240 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:20.363185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:20.363784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:20.369702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:20.370182 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:20.370654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:20.371335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:20.371781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:20.372261 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:20.508528 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:20.516569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d6a724dc-031d-478a-b5b6-2a6b3e6f8ab8 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.159040 Sep 18 01:01:20.517151 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 253/508] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:20 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:20.566200 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:20.566844 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:20.574365 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:20.575044 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:20.575367 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:20.576063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:20.576518 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:20.576946 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:20.638420 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:20.645990 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a76f60dd-3c5d-4949-922d-ea904a5cc5a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1292 microversion: 2.1 time: 0.084026 Sep 18 01:01:20.646576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 256/509] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:20 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1292 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:21.540685 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:21.541297 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:21.547994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:21.548483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:21.548922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:21.549647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:21.550108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:21.550565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:21.669095 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:21.669733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:21.676550 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:21.677189 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:21.677873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:21.678805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:21.680013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:21.680497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:21.688055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:21.697132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ddc5bc5d-5b6f-481b-839f-c45f6155db8a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.160406 Sep 18 01:01:21.698017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 254/510] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:21 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:21.841844 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:21.851683 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c63f71ef-8d1d-430a-9104-be3f305e44c5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.185941 Sep 18 01:01:21.852269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 257/511] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:21 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:22.508868 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-a006807b-11aa-4d0d-b519-55226205612a req-54c345f8-0edd-4b9c-a5d4-47402af9b462 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6c8469ab-e2ff-405e-9da7-e4e93e64fc72", "tag": "f0d60caf-e51b-4704-af82-ab86c2340a70"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:22.519234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-a006807b-11aa-4d0d-b519-55226205612a req-54c345f8-0edd-4b9c-a5d4-47402af9b462 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:22.519828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-a006807b-11aa-4d0d-b519-55226205612a req-54c345f8-0edd-4b9c-a5d4-47402af9b462 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:22.520337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-a006807b-11aa-4d0d-b519-55226205612a req-54c345f8-0edd-4b9c-a5d4-47402af9b462 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:22.533339 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-a006807b-11aa-4d0d-b519-55226205612a req-54c345f8-0edd-4b9c-a5d4-47402af9b462 service nova] Creating event network-changed:f0d60caf-e51b-4704-af82-ab86c2340a70 for instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 on hv-cinder-maste Sep 18 01:01:22.539602 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-a006807b-11aa-4d0d-b519-55226205612a req-54c345f8-0edd-4b9c-a5d4-47402af9b462 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035057 Sep 18 01:01:22.540393 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 255/512] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:22.722918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:22.723523 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:22.729942 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:22.730422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:22.730893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:22.731586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:22.732037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:22.732457 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:22.862561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:22.871243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cd451dae-6a0b-4861-a19b-4e6d4f843f06 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.152208 Sep 18 01:01:22.872827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 258/513] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:22 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:22.875878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:22.876578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:22.883556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:22.884036 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:22.884463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:22.885147 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:22.885718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:22.886145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:23.017643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:23.026003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7d0194a2-507c-47a9-84cc-8bc056238035 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.154789 Sep 18 01:01:23.026582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 256/514] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:22 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:23.894816 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:23.895415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:23.902003 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:23.902494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:23.902962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:23.903623 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:23.904065 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:23.904479 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:24.035620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:24.043927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-78828e43-2a0e-44c3-be70-59c6f3caefa6 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.153035 Sep 18 01:01:24.044487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 259/515] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:23 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:24.047457 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:24.048065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:24.054861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:24.055370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:24.055947 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:24.057030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:24.057561 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:24.058055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:24.189247 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:24.197655 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2feb8872-56a4-4dfa-9e09-9d3f5bba2c9d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.154074 Sep 18 01:01:24.198216 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 257/516] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:24 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:25.067809 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:25.068327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:25.075291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:25.075778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:25.076246 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:25.076929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:25.077376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:25.077794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:25.214241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:25.221581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:25.222275 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:25.224130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0243791d-8b6a-4762-ae5a-c277f64fadc0 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.160204 Sep 18 01:01:25.224718 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 260/517] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:25 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:25.228994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:25.229552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:25.230211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:25.231346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:25.231860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:25.232481 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:25.367314 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:25.374941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d22502fb-c87e-4447-b680-e6df5cbb52c8 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.157604 Sep 18 01:01:25.375514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 258/518] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:25 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:26.248782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:26.249382 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:26.255843 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:26.256312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:26.256739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:26.257595 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:26.258047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:26.258485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:26.392679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:26.399582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:26.400201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:26.401649 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fe8b7e18-674b-43bf-9ba0-ea84f2a3d84f tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.156697 Sep 18 01:01:26.402235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 261/519] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:26 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:26.406721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:26.407196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:26.407627 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:26.408315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:26.408783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:26.409465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:26.547150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:26.554899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-47e7e5aa-bbc6-42ad-8085-37b2d20ee6c2 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.159901 Sep 18 01:01:26.555467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 259/520] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:26 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:27.353047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-007a373d-0207-4bdb-bf11-8839eba6ef74 req-894dcc60-2675-485b-8f2e-b5f60787efa9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6c8469ab-e2ff-405e-9da7-e4e93e64fc72", "name": "network-vif-plugged", "status": "completed", "tag": "f0d60caf-e51b-4704-af82-ab86c2340a70"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:27.363820 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-007a373d-0207-4bdb-bf11-8839eba6ef74 req-894dcc60-2675-485b-8f2e-b5f60787efa9 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:27.364302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-007a373d-0207-4bdb-bf11-8839eba6ef74 req-894dcc60-2675-485b-8f2e-b5f60787efa9 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:27.364807 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-007a373d-0207-4bdb-bf11-8839eba6ef74 req-894dcc60-2675-485b-8f2e-b5f60787efa9 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:27.376427 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-007a373d-0207-4bdb-bf11-8839eba6ef74 req-894dcc60-2675-485b-8f2e-b5f60787efa9 service nova] Creating event network-vif-plugged:f0d60caf-e51b-4704-af82-ab86c2340a70 for instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 on hv-cinder-maste Sep 18 01:01:27.381300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-007a373d-0207-4bdb-bf11-8839eba6ef74 req-894dcc60-2675-485b-8f2e-b5f60787efa9 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032406 Sep 18 01:01:27.381944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 262/521] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:27.426701 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:27.427314 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:27.434225 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:27.434740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:27.435173 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:27.435860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:27.436340 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:27.436761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:27.574992 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:27.579633 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:27.580345 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:27.586172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4b243d82-45ee-4a49-b9be-583eb0c33382 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1737 microversion: 2.1 time: 0.163716 Sep 18 01:01:27.586821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:27.587305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:27.587764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:27.588032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 260/522] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:27 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1737 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:27.588479 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:27.588923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:27.589340 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:27.716988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:27.725239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3e2068cf-a2fc-4e92-8823-c58bfe21423f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.150173 Sep 18 01:01:27.725805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 263/523] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:27 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:28.609455 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:28.610072 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:28.616553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.617036 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.617472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:28.618164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.618649 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.619073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:28.747957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:28.748589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:28.749204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:28.755685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.756279 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.756405 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0711b2be-1ba6-4636-a6de-d5f7cbb9aaea tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1717 microversion: 2.1 time: 0.150820 Sep 18 01:01:28.756600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:28.756986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 261/524] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:28 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1717 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:28.757914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.758452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.758902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:28.884831 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:28.885534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:28.892091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.892667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.893096 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:28.893801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.894250 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.894893 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:28.898944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:28.903121 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:28.911907 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7bc7bdad-ea4b-4d00-9ca8-b7907872ba7d tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.167096 Sep 18 01:01:28.912796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 264/525] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:28 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:28.978093 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] HTTP exception thrown: Cannot 'detach_volume' instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c while it is in vm_state rescued Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Request method failure captured: Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: DELETE /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments/aecbfe01-aa3f-4ef9-b051-e26e7c98ea22 HTTP/1.1 Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzhkxRTrCUQQ4CHYpp5ib4E3J8zJRLpaPqCNiJzmchcAu_ipeHd4RwYdO1mlchW4uUJ4qXEZ8cWy5sMB5AwukkrGJE4mZR_1pkUaRSmGbYTXpM7QzG3OSBhI3tMi7AEO_3YHBoAPk5T6bb6rWvo2L-te49vgOqSNbUeL_AMq01aAlA Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:01:28.979723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_df01daf86854497fad1d1d016c269ef1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}] Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 54d4e437036640b58fda503c4d213f5c Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:01:28.981318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Cannot 'detach_volume' instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c while it is in vm_state rescued Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'server_id': '3e1c6d6a-9178-4c19-a032-ef79295fce8c', 'id': 'aecbfe01-aa3f-4ef9-b051-e26e7c98ea22'} Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:01:28.982242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c while it is in vm_state rescued Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c while it is in vm_state rescued Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:28.983834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Returning 409 to user: Cannot 'detach_volume' instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c while it is in vm_state rescued {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:01:28.984948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-396b9141-6ab0-4f23-86a0-264751824bc2 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments/aecbfe01-aa3f-4ef9-b051-e26e7c98ea22" status: 409 len: 152 microversion: 2.1 time: 0.101596 Sep 18 01:01:28.984948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 262/526] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:01:28 2022] DELETE /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments/aecbfe01-aa3f-4ef9-b051-e26e7c98ea22 => generated 152 bytes in 103 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 18 01:01:28.989982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:28.990660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:28.997117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.997590 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.998020 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:28.998721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:28.999170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:28.999588 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:29.063622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f8c2d69c-22e7-48ac-aafe-72a4a3eaf086 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "POST /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/action" status: 202 len: 0 microversion: 2.1 time: 0.077206 Sep 18 01:01:29.064269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 265/527] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:01:28 2022] POST /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/action => generated 0 bytes in 79 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:01:29.073119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:29.073726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:29.080181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:29.080670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:29.081107 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:29.081800 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:29.082268 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:29.082710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:29.213018 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:29.221244 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1cbd2c23-d6ee-4061-82ec-7f8b8215e701 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1725 microversion: 2.1 time: 0.152632 Sep 18 01:01:29.221808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 263/528] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:29 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:29.932394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:29.937337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:29.938034 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:29.944475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:29.944958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:29.945396 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:29.946079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:29.946557 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:29.946981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:30.085413 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:30.094929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f41545c5-62fb-4038-92b3-f4768a67cc27 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1366 microversion: 2.1 time: 0.163286 Sep 18 01:01:30.095566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 266/529] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:29 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:30.244915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:30.245514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:30.247401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:30.248046 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=10 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:30.254251 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:30.254755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:30.255188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:30.255872 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:30.256324 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:30.256741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:30.391396 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:30.401162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f312557c-b5b8-4226-8ec7-07e2479a4bda tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1725 microversion: 2.1 time: 0.159864 Sep 18 01:01:30.414487 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 264/530] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:30 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:31.120903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:31.121559 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:31.127972 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:31.128456 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:31.128890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:31.129577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:31.130029 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:31.130471 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:31.263708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:31.266136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:31.273742 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6793c069-2ad6-411e-b0be-dea97838bb5a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1554 microversion: 2.1 time: 0.156151 Sep 18 01:01:31.274311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 267/531] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:31 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1554 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:31.296277 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:31.296800 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:31.303263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:31.303745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:31.304243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:31.304946 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:31.305394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:31.305814 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:31.380517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3f3ca2bc-0294-4246-b782-b2a4a8143542 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.088035 Sep 18 01:01:31.381107 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 265/532] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 01:01:31 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-interface => generated 284 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:31.428851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:31.429514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:31.436541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:31.437023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:31.437454 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:31.438159 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:31.438629 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:31.439047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:31.568840 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:31.577504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e7fca852-f0cb-4d44-b536-970dbb736539 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1725 microversion: 2.1 time: 0.152580 Sep 18 01:01:31.578062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 268/533] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:31 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1725 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:32.319600 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-b849b404-b439-4a6c-9ff1-f2b8941203cb req-a31277f7-11e7-401c-9492-ae41c9c23ea3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6c8469ab-e2ff-405e-9da7-e4e93e64fc72", "tag": "f0d60caf-e51b-4704-af82-ab86c2340a70"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:32.329387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-b849b404-b439-4a6c-9ff1-f2b8941203cb req-a31277f7-11e7-401c-9492-ae41c9c23ea3 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:32.329962 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-b849b404-b439-4a6c-9ff1-f2b8941203cb req-a31277f7-11e7-401c-9492-ae41c9c23ea3 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:32.330488 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-b849b404-b439-4a6c-9ff1-f2b8941203cb req-a31277f7-11e7-401c-9492-ae41c9c23ea3 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:32.342156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-b849b404-b439-4a6c-9ff1-f2b8941203cb req-a31277f7-11e7-401c-9492-ae41c9c23ea3 service nova] Creating event network-changed:f0d60caf-e51b-4704-af82-ab86c2340a70 for instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 on hv-cinder-maste Sep 18 01:01:32.347374 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-b849b404-b439-4a6c-9ff1-f2b8941203cb req-a31277f7-11e7-401c-9492-ae41c9c23ea3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031930 Sep 18 01:01:32.347971 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 266/534] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:32.601941 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:32.602606 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:32.609042 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:32.609525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:32.609957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:32.610684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:32.611182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:32.611606 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:32.745144 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:32.753764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4b54d7af-da2d-4451-842c-8e245cd24c22 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1725 microversion: 2.1 time: 0.156369 Sep 18 01:01:32.754354 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 269/535] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:32 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1725 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:33.778111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:33.778956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:33.784998 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:33.785478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:33.785907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:33.786625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:33.787080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:33.787500 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:33.923167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:33.931991 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5b8f4197-27d0-4b92-b4be-475172f5b1ff tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1731 microversion: 2.1 time: 0.158587 Sep 18 01:01:33.932569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 267/536] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:33 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1731 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:33.954716 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:33.955199 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:33.961339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:33.961825 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:33.962350 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:33.962996 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:33.963503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:33.963883 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:34.040994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7f7d6dbf-4072-4503-9636-c7b54ce069be tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090379 Sep 18 01:01:34.041609 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 270/537] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 01:01:33 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:34.893063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-ec5c672f-37da-485c-bcc7-3a9c0aad8404 req-ef1566d1-e2e5-4000-86dd-a65bb6ec9f5d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e1c6d6a-9178-4c19-a032-ef79295fce8c", "tag": "bfcd0e79-9fb2-4117-b12a-c4de2d65b667"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:34.903141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ec5c672f-37da-485c-bcc7-3a9c0aad8404 req-ef1566d1-e2e5-4000-86dd-a65bb6ec9f5d service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:34.903610 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ec5c672f-37da-485c-bcc7-3a9c0aad8404 req-ef1566d1-e2e5-4000-86dd-a65bb6ec9f5d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:34.904035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ec5c672f-37da-485c-bcc7-3a9c0aad8404 req-ef1566d1-e2e5-4000-86dd-a65bb6ec9f5d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:34.915640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-ec5c672f-37da-485c-bcc7-3a9c0aad8404 req-ef1566d1-e2e5-4000-86dd-a65bb6ec9f5d service nova] Creating event network-changed:bfcd0e79-9fb2-4117-b12a-c4de2d65b667 for instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c on hv-cinder-maste Sep 18 01:01:34.922652 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-ec5c672f-37da-485c-bcc7-3a9c0aad8404 req-ef1566d1-e2e5-4000-86dd-a65bb6ec9f5d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033307 Sep 18 01:01:34.923293 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 268/538] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:36.936799 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-8a313763-4dc5-49c3-bb80-eb162924789f req-7e028778-a864-439d-9f34-50306b3018c1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e1c6d6a-9178-4c19-a032-ef79295fce8c", "tag": "bfcd0e79-9fb2-4117-b12a-c4de2d65b667"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:36.941634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=78,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:36.948688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-8a313763-4dc5-49c3-bb80-eb162924789f req-7e028778-a864-439d-9f34-50306b3018c1 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:36.949158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-8a313763-4dc5-49c3-bb80-eb162924789f req-7e028778-a864-439d-9f34-50306b3018c1 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:36.949576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-8a313763-4dc5-49c3-bb80-eb162924789f req-7e028778-a864-439d-9f34-50306b3018c1 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:36.962447 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-8a313763-4dc5-49c3-bb80-eb162924789f req-7e028778-a864-439d-9f34-50306b3018c1 service nova] Creating event network-changed:bfcd0e79-9fb2-4117-b12a-c4de2d65b667 for instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c on hv-cinder-maste Sep 18 01:01:36.967191 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-8a313763-4dc5-49c3-bb80-eb162924789f req-7e028778-a864-439d-9f34-50306b3018c1 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034666 Sep 18 01:01:36.967803 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 271/539] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:39.403639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:39.404241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:39.406339 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=4 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:39.413553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:39.414684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:39.415116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:39.415810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:39.416253 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:39.416667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:39.448080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ffe3445f-5c0d-4d9d-9373-1257a1904f8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments/e7bc6d89-71b1-435d-815c-bfe81a6b925d" status: 200 len: 194 microversion: 2.20 time: 0.049699 Sep 18 01:01:39.448892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 269/540] 10.222.0.42 () {60 vars in 1454 bytes} [Sun Sep 18 01:01:39 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments/e7bc6d89-71b1-435d-815c-bfe81a6b925d => generated 194 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 01:01:39.520059 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:39.520633 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:39.527284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:39.527762 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:39.528188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:39.528873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:39.529319 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:39.529738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:39.656588 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lazy-loading 'services' on Instance uuid 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:01:39.772467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-31434c40-242b-44fa-93c0-9dd4794d6e52 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments/e7bc6d89-71b1-435d-815c-bfe81a6b925d" status: 202 len: 0 microversion: 2.20 time: 0.257108 Sep 18 01:01:39.773680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 272/541] 10.222.0.42 () {60 vars in 1457 bytes} [Sun Sep 18 01:01:39 2022] DELETE /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments/e7bc6d89-71b1-435d-815c-bfe81a6b925d => generated 0 bytes in 259 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:01:39.782233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:39.784546 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:39.790965 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:39.791524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:39.792068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:39.792937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:39.793407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:39.793895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:39.826142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c9c900b2-7490-4092-8963-d7bd66b3030f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047900 Sep 18 01:01:39.826910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 270/542] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 18 01:01:39 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 01:01:40.844517 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:40.846866 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:40.853991 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:40.854518 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:40.854991 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:40.855724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:40.856219 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:40.856698 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:40.889470 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-330c826b-e45a-4e06-ac2f-af90da794335 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053083 Sep 18 01:01:40.890026 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 273/543] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 18 01:01:40 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 01:01:41.902428 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:41.904578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:41.911133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:41.911613 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:41.912044 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:41.912732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:41.913183 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:41.914218 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:41.944946 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0976cd24-066b-4074-84f7-4fefab75291f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.046954 Sep 18 01:01:41.945534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 271/544] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 18 01:01:41 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:42.013022 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:42.013635 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:42.020053 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:42.020534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:42.020964 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:42.021641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:42.022087 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:42.022536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:42.048917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:01:42.133214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8f22dcf2-7de5-4f45-85bd-394a57a9d7b5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 204 len: 0 microversion: 2.20 time: 0.124646 Sep 18 01:01:42.133970 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 274/545] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 18 01:01:42 2022] DELETE /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 01:01:42.144088 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:42.144707 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:42.151663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:42.152150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:42.152634 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:42.153303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:42.153833 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:42.154304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:42.292639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:42.300823 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:42.302271 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:42.303467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5437c0eb-a586-4213-bee9-77844c1ac238 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.164037 Sep 18 01:01:42.304023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 272/546] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:42 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:43.329962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:43.331066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:43.339273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:43.339869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:43.340470 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:43.341394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:43.341989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:43.342644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:43.477969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:43.486291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:43.487780 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:43.488962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3057c549-ca61-4dd3-a5ae-59a6c8680fa6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.163650 Sep 18 01:01:43.489530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 275/547] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:43 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:44.517971 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:44.519056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:44.526273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:44.526630 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:44.527100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:44.527779 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:44.528235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:44.529238 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:44.665170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:44.673331 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:44.674801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:44.675956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7f66ad5b-ac29-4b9e-9826-654c589a54a8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 200 len: 1724 microversion: 2.20 time: 0.163013 Sep 18 01:01:44.676510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 273/548] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:44 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 1724 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:45.346431 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-8b99884b-756a-415c-a741-40258a385d13 req-59b07b9c-93a6-4a3c-9149-3ed0ffdea99b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5efa25bd-04ff-49f6-9fd5-6e8c82bcd152", "name": "network-vif-deleted", "tag": "af5eca8e-a823-4591-9077-2a4b9b7fb155"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:45.357539 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-8b99884b-756a-415c-a741-40258a385d13 req-59b07b9c-93a6-4a3c-9149-3ed0ffdea99b service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:45.358047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-8b99884b-756a-415c-a741-40258a385d13 req-59b07b9c-93a6-4a3c-9149-3ed0ffdea99b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:45.358486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-8b99884b-756a-415c-a741-40258a385d13 req-59b07b9c-93a6-4a3c-9149-3ed0ffdea99b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:45.371023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-8b99884b-756a-415c-a741-40258a385d13 req-59b07b9c-93a6-4a3c-9149-3ed0ffdea99b service nova] Creating event network-vif-deleted:af5eca8e-a823-4591-9077-2a4b9b7fb155 for instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 on hv-cinder-maste Sep 18 01:01:45.377499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-8b99884b-756a-415c-a741-40258a385d13 req-59b07b9c-93a6-4a3c-9149-3ed0ffdea99b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034422 Sep 18 01:01:45.378496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 276/549] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:45.702220 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:45.702946 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:45.710263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:45.711461 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:45.711928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:45.712662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:45.713152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:45.713579 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:45.727320 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] HTTP exception thrown: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Request method failure captured: Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 HTTP/1.1 Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzmaso6D5zTG4pZx5n1a7YeINaSoMRjyPDFIzt6YRAF1nyLLD5VG506tTehjPTjn5tXUtJYrbv0QwuLl0EK5wvD84ApKctiQlJ_EQaLnOIgCLEQVD82q7MrE8lKvBj3E-XoYpBdI5Uua4R9pUyQ3IfLhhX05m1BgPYuSan8QhxRxnw Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:01:45.729075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}] Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 294ae98a8492468893beb3b8c2870d5e Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:01:45.731302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '5efa25bd-04ff-49f6-9fd5-6e8c82bcd152'} Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:01:45.732125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:01:45.733995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:01:45.735186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Returning 404 to user: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:01:45.735186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2017e254-557d-4390-a16c-bf304ea2a658 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 404 len: 111 microversion: 2.20 time: 0.032970 Sep 18 01:01:45.735186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 274/550] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:45 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:01:45.741725 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a7420989-4d0d-4331-8880-f4c22acf4962 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '.version_select at 0x7fcab1ca7af0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:45.760940 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a7420989-4d0d-4331-8880-f4c22acf4962 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2071684712" status: 204 len: 0 microversion: 2.20 time: 0.023438 Sep 18 01:01:45.761669 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 277/551] 10.222.0.42 () {60 vars in 1262 bytes} [Sun Sep 18 01:01:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2071684712 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 01:01:46.448311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ebb91cfb-a6c2-4f93-bb05-05b4c34fea35 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'create', calling method: .version_select at 0x7fcab1c84700>, body: {"keypair": {"name": "tempest-keypair-1218396652", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEfeIvcga5rvD3Wzol3B8/ELrMpuNqPpjwIKmaEz6nj1uoM53nlC8IDsvC2YemOo7+oEKqGprzxqFZTWgqRb2e+8HCr4RHSmzWvGB3d+Mfd5WhHIRK5dvs6db5De2X8rgQ=="}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:46.465503 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-ebb91cfb-a6c2-4f93-bb05-05b4c34fea35 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for project 505bd70415a04f53a64645eb7959a5fb. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:01:46.469887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-ebb91cfb-a6c2-4f93-bb05-05b4c34fea35 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for user 294ae98a8492468893beb3b8c2870d5e and project 505bd70415a04f53a64645eb7959a5fb. Resources: {'key_pairs'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:01:46.470546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:46.472886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:46.480358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:46.480851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:46.481287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:46.482000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:46.483238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:46.483681 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:46.496064 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ebb91cfb-a6c2-4f93-bb05-05b4c34fea35 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.052471 Sep 18 01:01:46.496658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 275/552] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Sep 18 01:01:46 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 54 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Sep 18 01:01:46.581360 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-dbcc7893-8996-4b4d-8330-554a4579d7e0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/action" status: 202 len: 0 microversion: 2.1 time: 0.115030 Sep 18 01:01:46.587063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 278/553] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:01:46 2022] POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/action => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:01:46.593438 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:46.594145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:46.602941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:46.603318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:46.604024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:46.604780 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:46.605224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:46.605665 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:46.819273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:46.831992 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d5b39ee2-033f-48b0-a892-3046b67439c7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1749 microversion: 2.1 time: 0.243197 Sep 18 01:01:46.832792 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 276/554] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:46 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1749 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:47.862853 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:47.863565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:47.869984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:47.870498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:47.870989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:47.871700 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:47.872184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:47.872662 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:48.158848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:48.167847 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bba68932-c0d0-4b8d-bdd3-0359927a7951 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.313266 Sep 18 01:01:48.168491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 279/555] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:47 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:48.374359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-864427118", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "7ab8f07f-d17d-470a-a9f4-d9a359c2132d"}], "security_groups": [{"name": "tempest-securitygroup--904744984"}], "key_name": "tempest-keypair-1218396652", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:48.493414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] validate_networks() for [('7ab8f07f-d17d-470a-a9f4-d9a359c2132d', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:01:48.913173 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:01:48.914237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:01:48.914875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:01:48.915478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:01:48.916085 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:01:48.930493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:48.931346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:48.932140 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:48.933589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:48.934406 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:48.935213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:48.973292 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for project 505bd70415a04f53a64645eb7959a5fb. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:01:48.984755 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Getting quotas for user 294ae98a8492468893beb3b8c2870d5e and project 505bd70415a04f53a64645eb7959a5fb. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:01:49.010153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:01:49.015142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:01:49.102678 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ed2c6b00-974d-466c-9141-8d09d0a3262a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.733671 Sep 18 01:01:49.103435 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 277/556] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Sep 18 01:01:48 2022] POST /compute/v2.1/servers => generated 401 bytes in 737 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 18 01:01:49.116078 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-eb3a41e5-e473-4267-b095-95bba94e8bf7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:49.116958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-eb3a41e5-e473-4267-b095-95bba94e8bf7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:49.192102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:49.192728 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:49.200839 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:49.201829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:49.202451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:49.203187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:49.203647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:49.204330 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:49.235687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-eb3a41e5-e473-4267-b095-95bba94e8bf7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:49.243682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-eb3a41e5-e473-4267-b095-95bba94e8bf7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:49.245117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-eb3a41e5-e473-4267-b095-95bba94e8bf7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:49.246765 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-eb3a41e5-e473-4267-b095-95bba94e8bf7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1281 microversion: 2.20 time: 0.134531 Sep 18 01:01:49.246975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 280/557] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:49 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1281 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:49.366899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:49.375415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-21fe1dd1-9207-4074-a300-fb1655c55fc5 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.188176 Sep 18 01:01:49.381042 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 278/558] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:49 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:50.271158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:50.271875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:50.279160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:50.279639 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:50.280065 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:50.280784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:50.281228 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:50.281660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:50.344353 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:50.352940 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:50.354389 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:50.355514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-480a1a29-e85f-4379-9e68-4a443ac9da51 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1333 microversion: 2.20 time: 0.088274 Sep 18 01:01:50.356112 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 281/559] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:50 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1333 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:50.401175 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:50.401773 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:50.408263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:50.408740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:50.409170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:50.409886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:50.410336 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:50.417610 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:50.576713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:50.586149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e19d6674-7e9a-44aa-a1b6-e9b5f76140fa tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.189325 Sep 18 01:01:50.587412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 279/560] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:50 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:51.381467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:51.382205 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:51.389729 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:51.390732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:51.391262 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:51.392419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:51.392882 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:51.393477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:51.463048 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:51.472913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:51.475087 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:51.476759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2f101591-25af-4915-be6b-3d162e3369eb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1339 microversion: 2.20 time: 0.099278 Sep 18 01:01:51.477430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 282/561] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:51 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1339 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:51.612840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:51.614168 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:51.623032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:51.623534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:51.623970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:51.624749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:51.625305 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:51.625896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:51.778676 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:51.787604 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-986a8ce7-3a82-48a2-9b25-ebeb5989db48 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.179755 Sep 18 01:01:51.788997 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 280/562] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:51 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:01:52.324295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-07ad4bb0-f780-476f-b4fb-7bb32fd6669c req-73283272-1d2f-4277-a892-b758e8ce7d04 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8a40bb3d-0090-4ce2-9782-c61f0161e064", "tag": "5025d940-e81a-4c19-8fda-e4fac2888b2a"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:52.349487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-07ad4bb0-f780-476f-b4fb-7bb32fd6669c req-73283272-1d2f-4277-a892-b758e8ce7d04 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:52.350065 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-07ad4bb0-f780-476f-b4fb-7bb32fd6669c req-73283272-1d2f-4277-a892-b758e8ce7d04 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:52.350584 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-07ad4bb0-f780-476f-b4fb-7bb32fd6669c req-73283272-1d2f-4277-a892-b758e8ce7d04 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:52.363503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-07ad4bb0-f780-476f-b4fb-7bb32fd6669c req-73283272-1d2f-4277-a892-b758e8ce7d04 service nova] Creating event network-changed:5025d940-e81a-4c19-8fda-e4fac2888b2a for instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 on hv-cinder-maste Sep 18 01:01:52.368634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-07ad4bb0-f780-476f-b4fb-7bb32fd6669c req-73283272-1d2f-4277-a892-b758e8ce7d04 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048162 Sep 18 01:01:52.369455 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 283/563] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:52.510428 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:52.511082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:52.519642 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:52.520139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:52.520592 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:52.521295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:52.521745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:52.522163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:52.712141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:52.720584 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:52.722011 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:52.723133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ffcfffb2-9065-41fc-8a80-d8031af0d0ff tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1406 microversion: 2.20 time: 0.216937 Sep 18 01:01:52.723990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 281/564] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:52 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1406 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:52.812979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:52.813684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:52.820138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:52.820626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:52.821064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:52.821771 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:52.822252 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:52.822755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:52.976139 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:52.985658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d4b0df19-7810-4fcb-80b5-a5c7fd171638 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.177414 Sep 18 01:01:52.986223 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 284/565] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:52 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:53.748507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:53.749104 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:53.755623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:53.756101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:53.756531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:53.757273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:53.758268 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:53.758723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:53.897482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:53.899813 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:53.907840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:53.909301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:53.910487 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-609e6e01-fd66-4b03-93e2-a9d83ff37617 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1406 microversion: 2.20 time: 0.165844 Sep 18 01:01:53.911054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 282/566] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:53 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1406 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:54.010232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:54.011592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:54.019715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:54.020218 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:54.020635 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:54.021378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:54.021806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:54.022227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:54.038221 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:54.179720 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:54.187915 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5dac54b5-6008-459e-b199-0aa9be6d6ed4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.182304 Sep 18 01:01:54.188489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 285/567] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:54 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:54.936343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:54.936942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:54.943556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:54.944068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:54.944501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:54.945262 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:54.945736 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:54.946158 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:55.078667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:55.087607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:55.089083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:55.090945 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f7cb9707-790a-47fb-9df2-7131beb70afb tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1406 microversion: 2.20 time: 0.157818 Sep 18 01:01:55.090945 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 283/568] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:54 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1406 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:01:55.213278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:55.213948 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:55.220981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:55.221469 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:55.221906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:55.222672 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:55.223130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:55.223550 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:55.373271 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:55.382349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0d8b16dd-26da-4979-bd64-e8406a9e368c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.173569 Sep 18 01:01:55.383274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 286/569] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:55 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:56.115557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:56.116257 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:56.122866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:56.123371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:56.123806 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:56.124520 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:56.124978 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:56.125407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:56.267912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:56.277134 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:56.278623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:56.279745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-09aa2f7c-15c4-4b10-bdb9-52f21d42ae53 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1406 microversion: 2.20 time: 0.168118 Sep 18 01:01:56.280299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 284/570] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:56 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1406 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:56.406083 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:56.406712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:56.413244 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:56.413784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:56.414226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:56.414981 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:56.415460 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:56.415888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:56.576497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:56.585107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c56d8d82-c6db-490f-add1-0c0be6466cd8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.182868 Sep 18 01:01:56.586356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 287/571] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:56 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:57.305131 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:57.306035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:57.312625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:57.313101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:57.313533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:57.314215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:57.314693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:57.315109 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:57.459130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:57.470110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:57.471972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:57.473120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-10e2f600-0406-4b4f-b460-5274baca973d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1406 microversion: 2.20 time: 0.172562 Sep 18 01:01:57.473685 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 285/572] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:57 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1406 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:57.523507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-30d9cdf8-bfa3-4721-9c75-3efbb55e3f66 req-b46a4226-f885-4708-bf9e-31f65b5839a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a40bb3d-0090-4ce2-9782-c61f0161e064", "name": "network-vif-plugged", "status": "completed", "tag": "5025d940-e81a-4c19-8fda-e4fac2888b2a"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:01:57.534774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-30d9cdf8-bfa3-4721-9c75-3efbb55e3f66 req-b46a4226-f885-4708-bf9e-31f65b5839a2 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:57.535288 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-30d9cdf8-bfa3-4721-9c75-3efbb55e3f66 req-b46a4226-f885-4708-bf9e-31f65b5839a2 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:57.535682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-30d9cdf8-bfa3-4721-9c75-3efbb55e3f66 req-b46a4226-f885-4708-bf9e-31f65b5839a2 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:57.548218 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-30d9cdf8-bfa3-4721-9c75-3efbb55e3f66 req-b46a4226-f885-4708-bf9e-31f65b5839a2 service nova] Creating event network-vif-plugged:5025d940-e81a-4c19-8fda-e4fac2888b2a for instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 on hv-cinder-maste Sep 18 01:01:57.553268 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-30d9cdf8-bfa3-4721-9c75-3efbb55e3f66 req-b46a4226-f885-4708-bf9e-31f65b5839a2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033901 Sep 18 01:01:57.554715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 288/573] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:01:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:01:57.612232 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:57.619577 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:57.628196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:57.628749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:57.629237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:57.630002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:57.630536 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:57.631038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:57.818316 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:57.826944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5aa0db2b-ffce-4bd2-8b2b-3560cb5c1548 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.220911 Sep 18 01:01:57.827513 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 286/574] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:57 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:58.499614 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:58.500261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:58.507724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:58.508266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:58.508718 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:58.509503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:58.510010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:58.510493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:58.663185 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:58.671788 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:58.673210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:58.674320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c6484870-e8bc-4f96-aeb4-8b7762536212 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1406 microversion: 2.20 time: 0.179472 Sep 18 01:01:58.674901 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 289/575] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:58 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1406 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:01:58.850831 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:58.851455 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:58.858142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:58.858821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:58.859146 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:58.859852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:58.860308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:58.860733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:58.951689 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:58.997215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=10 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:59.022414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:59.030757 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ea38254e-ff97-49c3-955d-d3dc8e68e479 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.183913 Sep 18 01:01:59.031328 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 287/576] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:01:58 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:01:59.699447 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:01:59.700278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:01:59.702609 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:01:59.709800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:59.710796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:59.711224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:59.711950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:01:59.712412 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:01:59.712829 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:01:59.857524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:59.870413 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:59.873432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:01:59.875613 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ef8829dd-6454-4b63-923a-a6587f97fca3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1406 microversion: 2.20 time: 0.180111 Sep 18 01:01:59.876276 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 290/577] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:01:59 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1406 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:02:00.054667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:00.055278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:00.062919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:00.063525 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:00.064139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:00.064944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:00.065414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:00.065849 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:00.222978 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:00.235910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-01088f1d-9234-400f-9221-5cd696dc7f06 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.185591 Sep 18 01:02:00.236617 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 288/578] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:00 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:00.914478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:00.915110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:00.921881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:00.922367 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:00.922849 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:00.923555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:00.924006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:00.924450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:01.076226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:01.089778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:01.090182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:01.091450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0e56d6d5-c5fd-498c-88d2-c8724ad9bda1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1599 microversion: 2.20 time: 0.181423 Sep 18 01:02:01.092038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 291/579] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:00 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1599 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:02:01.117814 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:01.118392 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:01.125814 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:01.126415 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:01.126995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:01.127801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:01.128341 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:01.128847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:01.213877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b3b7df0b-f379-434a-9edd-074986cd2474 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.099879 Sep 18 01:02:01.214931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 289/580] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Sep 18 01:02:01 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-interface => generated 285 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 01:02:01.265538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:01.266160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:01.273902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:01.274530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:01.275002 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:01.275783 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:01.276243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:01.276669 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:01.570786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:01.579786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6e3af91c-e373-4698-a336-71f7609e2592 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.323646 Sep 18 01:02:01.580377 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 292/581] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:01 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:02:02.288445 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-07229fd4-da90-4d2b-9a67-c95a44c42623 req-013c098a-bf5d-4557-ab1c-6015038d4ccc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8a40bb3d-0090-4ce2-9782-c61f0161e064", "tag": "5025d940-e81a-4c19-8fda-e4fac2888b2a"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:02.298943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-07229fd4-da90-4d2b-9a67-c95a44c42623 req-013c098a-bf5d-4557-ab1c-6015038d4ccc service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:02.299427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-07229fd4-da90-4d2b-9a67-c95a44c42623 req-013c098a-bf5d-4557-ab1c-6015038d4ccc service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:02.299858 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-07229fd4-da90-4d2b-9a67-c95a44c42623 req-013c098a-bf5d-4557-ab1c-6015038d4ccc service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:02.311464 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-07229fd4-da90-4d2b-9a67-c95a44c42623 req-013c098a-bf5d-4557-ab1c-6015038d4ccc service nova] Creating event network-changed:5025d940-e81a-4c19-8fda-e4fac2888b2a for instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 on hv-cinder-maste Sep 18 01:02:02.317120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-07229fd4-da90-4d2b-9a67-c95a44c42623 req-013c098a-bf5d-4557-ab1c-6015038d4ccc service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032866 Sep 18 01:02:02.317746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 290/582] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:02.603585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:02.604393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:02.612462 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:02.612961 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:02.613398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:02.614114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:02.614593 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:02.615018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:02.786852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:02.795914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8b566bf3-108b-442d-a7dd-f84738e0d3b9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.196301 Sep 18 01:02:02.796721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 293/583] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:02 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:03.822359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:03.823013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:03.830003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:03.830533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:03.830968 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:03.831684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:03.832137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:03.832560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:04.016477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:04.025959 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-585da26b-12f0-4d1c-8822-410dec7c12b4 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1754 microversion: 2.1 time: 0.209162 Sep 18 01:02:04.026801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 291/584] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:03 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1754 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:05.050622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:05.051326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:05.057908 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:05.058525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:05.058995 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:05.059910 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:05.060504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:05.061071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:05.217366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:05.225833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ef5ee1e0-0f4a-4d1e-b360-8063cee5cdf9 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1757 microversion: 2.1 time: 0.179555 Sep 18 01:02:05.226986 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 294/585] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:05 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1757 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:15.787578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=67,nova_api:INSERT=4 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:15.788236 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=49 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:15.841713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:15.842231 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:15.848208 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:15.848690 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:15.849126 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:15.849811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:15.850262 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:15.850718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:15.875050 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3fef2021-54fd-4470-8bd7-8ca107472071 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/ips" status: 200 len: 148 microversion: 2.1 time: 0.093666 Sep 18 01:02:15.875596 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 292/586] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:02:15 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/ips => generated 148 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:16.206742 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:16.207262 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:16.210177 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=55,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:16.211419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=53 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:16.217330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:16.217813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:16.218247 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:16.218969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:16.219425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:16.219852 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:16.306423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lazy-loading 'services' on Instance uuid 3e1c6d6a-9178-4c19-a032-ef79295fce8c {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:16.379257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e690ec0c-cf30-4a4a-abf3-50bd22181961 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments/aecbfe01-aa3f-4ef9-b051-e26e7c98ea22" status: 202 len: 0 microversion: 2.1 time: 0.176814 Sep 18 01:02:16.379816 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 295/587] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:02:16 2022] DELETE /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments/aecbfe01-aa3f-4ef9-b051-e26e7c98ea22 => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:02:16.388930 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:16.391256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:16.397896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:16.398397 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:16.398864 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:16.399743 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:16.400120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:16.400618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:16.431734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-dfb60c54-d036-4e7d-b5a6-e42791e4cd44 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046834 Sep 18 01:02:16.432299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 293/588] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:02:16 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:17.240936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "29874eb5-2925-4cdb-a3fd-5eaa88a89340"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:17.243525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:17.250004 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.250442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.250885 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.251597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.252055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.252478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.445390 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:17.447694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:17.459278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.459855 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.460879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.461672 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.462147 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.462602 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.498775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-43adc1c8-ea2c-42b8-b93d-05a681398a60 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.057870 Sep 18 01:02:17.499424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 294/589] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:02:17 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c/os-volume_attachments => generated 25 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 18 01:02:17.573948 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-49e0346c-213d-4951-b2fa-06f639776485 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.337692 Sep 18 01:02:17.574791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 296/590] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:02:17 2022] POST /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments => generated 194 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:02:17.598747 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:17.599523 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:17.606875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.607371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.607809 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.608526 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.609163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.609597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.640752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:02:17.736032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e8fc6920-54ae-494e-886f-eb72a5286542 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 204 len: 0 microversion: 2.1 time: 0.140860 Sep 18 01:02:17.736732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 295/591] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:02:17 2022] DELETE /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 18 01:02:17.747036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:17.747769 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:17.754750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.755872 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.756301 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.757100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:17.758291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:17.758832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:17.904952 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:17.913243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a428263b-3375-48d6-a59e-74fbdb44af5b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1691 microversion: 2.1 time: 0.170351 Sep 18 01:02:17.913844 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 297/592] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:17 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1691 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:18.937216 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:18.937910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:18.944311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:18.945379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:18.945917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:18.946725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:18.947182 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:18.947602 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:19.077646 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:19.086295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-62c06fe6-2eb0-4d1d-9bd3-c696071b81aa tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1691 microversion: 2.1 time: 0.152672 Sep 18 01:02:19.086892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 296/593] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:18 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1691 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:20.109152 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:20.109809 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:20.116270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:20.117237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:20.117663 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:20.118339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:20.118828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:20.119249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:20.246928 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:20.254876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-50154775-6955-47bf-93fd-3c7e19a1de4c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1691 microversion: 2.1 time: 0.149496 Sep 18 01:02:20.255453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 298/594] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:20 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1691 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:21.277148 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:21.278650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:21.285605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:21.286452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:21.286942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:21.287670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:21.288165 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:21.289119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:21.638292 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:21.646178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-92356f34-0501-4018-833d-b5447a3fc1ef tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1691 microversion: 2.1 time: 0.372916 Sep 18 01:02:21.646773 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 297/595] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:21 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1691 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:22.380651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-a584a362-78e9-468d-b11b-640b9a2dd32e req-1b614880-2033-43c0-8f7c-26a6a2050f05 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e1c6d6a-9178-4c19-a032-ef79295fce8c", "name": "network-vif-deleted", "tag": "bfcd0e79-9fb2-4117-b12a-c4de2d65b667"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:22.397261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-a584a362-78e9-468d-b11b-640b9a2dd32e req-1b614880-2033-43c0-8f7c-26a6a2050f05 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:22.397865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-a584a362-78e9-468d-b11b-640b9a2dd32e req-1b614880-2033-43c0-8f7c-26a6a2050f05 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:22.398379 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-a584a362-78e9-468d-b11b-640b9a2dd32e req-1b614880-2033-43c0-8f7c-26a6a2050f05 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:22.413986 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-a584a362-78e9-468d-b11b-640b9a2dd32e req-1b614880-2033-43c0-8f7c-26a6a2050f05 service nova] Creating event network-vif-deleted:bfcd0e79-9fb2-4117-b12a-c4de2d65b667 for instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c on hv-cinder-maste Sep 18 01:02:22.419636 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-a584a362-78e9-468d-b11b-640b9a2dd32e req-1b614880-2033-43c0-8f7c-26a6a2050f05 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045229 Sep 18 01:02:22.419957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 299/596] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:02:22.670129 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:22.670830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:22.677690 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:22.678190 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:22.678763 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:22.680107 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:22.680649 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:22.681146 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:22.722781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:22.723340 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:22.723902 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:22.724352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lazy-loading 'fault' on Instance uuid 3e1c6d6a-9178-4c19-a032-ef79295fce8c {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:22.771792 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:22.780321 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1ec0e8e0-5a01-486a-932e-009ecf1dd074 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 200 len: 1336 microversion: 2.1 time: 0.114013 Sep 18 01:02:22.781033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 298/597] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:22 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 1336 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:02:23.805979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:23.806707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 3e1c6d6a-9178-4c19-a032-ef79295fce8c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:23.813436 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:23.813912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:23.814355 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:23.815085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:23.815557 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:23.815976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:23.829740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] HTTP exception thrown: Instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c could not be found. Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Request method failure captured: Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c HTTP/1.1 Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzhkxRTrCUQQ4CHYpp5ib4E3J8zJRLpaPqCNiJzmchcAu_ipeHd4RwYdO1mlchW4uUJ4qXEZ8cWy5sMB5AwukkrGJE4mZR_1pkUaRSmGbYTXpM7QzG3OSBhI3tMi7AEO_3YHBoAPk5T6bb6rWvo2L-te49vgOqSNbUeL_AMq01aAlA Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:02:23.831626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_df01daf86854497fad1d1d016c269ef1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}] Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 54d4e437036640b58fda503c4d213f5c Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c could not be found. Sep 18 01:02:23.833162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '3e1c6d6a-9178-4c19-a032-ef79295fce8c'} Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c could not be found. Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:23.834126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c could not be found. Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c could not be found. Sep 18 01:02:23.835704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:23.837027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Returning 404 to user: Instance 3e1c6d6a-9178-4c19-a032-ef79295fce8c could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:23.837027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d8bea0b2-a9a0-448c-8d89-daab413c0c39 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c" status: 404 len: 111 microversion: 2.1 time: 0.031764 Sep 18 01:02:23.837027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 300/598] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:23 2022] GET /compute/v2.1/servers/3e1c6d6a-9178-4c19-a032-ef79295fce8c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:23.841738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-55b74d38-cba1-4ad3-b984-3f2aade0504b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '.version_select at 0x7fcab1d4b1f0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:23.862011 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-55b74d38-cba1-4ad3-b984-3f2aade0504b tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1235240290" status: 202 len: 0 microversion: 2.1 time: 0.023900 Sep 18 01:02:23.862570 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 299/599] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Sep 18 01:02:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1235240290 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:02:26.708213 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:26.708727 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:26.716224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:26.716714 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:26.717147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:26.717894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:26.718344 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:26.718801 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:26.746487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:02:26.821480 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3a9d4341-1988-452b-9c78-3fe248ff01c3 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "DELETE /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 204 len: 0 microversion: 2.1 time: 0.117154 Sep 18 01:02:26.822063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 301/600] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:02:26 2022] DELETE /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:02:26.830335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:26.830981 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:26.838180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:26.838705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:26.839142 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:26.839839 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:26.840290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:26.840712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:26.968713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:26.977086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9474c115-6aba-4cf3-89ea-5fc0c18f7dcd tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1508 microversion: 2.1 time: 0.150953 Sep 18 01:02:26.977656 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 300/601] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:26 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1508 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:28.001531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:28.002223 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:28.009866 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:28.010488 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:28.010951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:28.011633 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:28.012085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:28.012606 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:28.150622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:28.158539 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6ccddc89-39fc-461f-889f-b6c2407f3584 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1508 microversion: 2.1 time: 0.161261 Sep 18 01:02:28.159127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 302/602] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:27 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1508 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:29.190789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:29.191395 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:29.198331 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:29.198838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:29.199270 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:29.199956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:29.200402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:29.200823 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:29.333848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:29.342328 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3d93c18d-a5f0-4551-914f-6cad035a1209 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1508 microversion: 2.1 time: 0.155324 Sep 18 01:02:29.342939 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 301/603] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:29 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1508 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:30.365558 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:30.366898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:30.374051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.374707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.375329 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.376100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.376633 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.377179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.477685 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "29874eb5-2925-4cdb-a3fd-5eaa88a89340"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:30.480860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:30.487450 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.487914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.488353 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.489059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.489510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.489923 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.499005 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:30.508098 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f9fc6944-c4a4-4a43-9090-fb2c232a96bb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1466 microversion: 2.1 time: 0.147521 Sep 18 01:02:30.509249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 303/604] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:30 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1466 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:30.592225 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-50936462-3a4a-4aa5-aee9-e60268b779a8 req-c4b75292-5f66-4084-909c-0fbc2372f828 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "46b96d5f-1930-4b82-afcb-62726307e170", "name": "network-vif-deleted", "tag": "d0ec5e41-6756-47f2-a199-19f6930aa09a"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:30.594547 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Invalid volume: volume 29874eb5-2925-4cdb-a3fd-5eaa88a89340 is already attached to instances: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: POST /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments HTTP/1.1 Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 74 Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:02:30.596912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {"volumeAttachment": {"volumeId": "29874eb5-2925-4cdb-a3fd-5eaa88a89340"}} Sep 18 01:02:30.598622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Invalid volume: volume 29874eb5-2925-4cdb-a3fd-5eaa88a89340 is already attached to instances: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'server_id': '6c8469ab-e2ff-405e-9da7-e4e93e64fc72', 'body': {'volumeAttachment': {'volumeId': '29874eb5-2925-4cdb-a3fd-5eaa88a89340'}}} Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 29874eb5-2925-4cdb-a3fd-5eaa88a89340 is already attached to instances: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:30.599534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 29874eb5-2925-4cdb-a3fd-5eaa88a89340 is already attached to instances: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:30.600877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:30.603605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:30.603605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:30.603605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:30.603605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 29874eb5-2925-4cdb-a3fd-5eaa88a89340 is already attached to instances: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 Sep 18 01:02:30.603605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:30.603605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 400 to user: Invalid volume: volume 29874eb5-2925-4cdb-a3fd-5eaa88a89340 is already attached to instances: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:30.603605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a6a79cbd-3bcb-4c2d-b76f-61d2b4b3d309 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.125489 Sep 18 01:02:30.610030 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-50936462-3a4a-4aa5-aee9-e60268b779a8 req-c4b75292-5f66-4084-909c-0fbc2372f828 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.610740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-50936462-3a4a-4aa5-aee9-e60268b779a8 req-c4b75292-5f66-4084-909c-0fbc2372f828 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.611167 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-50936462-3a4a-4aa5-aee9-e60268b779a8 req-c4b75292-5f66-4084-909c-0fbc2372f828 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.615883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 302/605] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:02:30 2022] POST /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments => generated 174 bytes in 133 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 18 01:02:30.624335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-50936462-3a4a-4aa5-aee9-e60268b779a8 req-c4b75292-5f66-4084-909c-0fbc2372f828 service nova] Creating event network-vif-deleted:d0ec5e41-6756-47f2-a199-19f6930aa09a for instance 46b96d5f-1930-4b82-afcb-62726307e170 on hv-cinder-maste Sep 18 01:02:30.629525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-50936462-3a4a-4aa5-aee9-e60268b779a8 req-c4b75292-5f66-4084-909c-0fbc2372f828 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.118887 Sep 18 01:02:30.633912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 304/606] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:30.669674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:30.670153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:30.678989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.679458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.679925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.680626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.681073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.681734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.766330 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lazy-loading 'services' on Instance uuid 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:30.847015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ef9eae3d-1262-4b6e-b372-a2cc1eaf5e7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments/29874eb5-2925-4cdb-a3fd-5eaa88a89340" status: 202 len: 0 microversion: 2.1 time: 0.181017 Sep 18 01:02:30.848851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 303/607] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:02:30 2022] DELETE /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments/29874eb5-2925-4cdb-a3fd-5eaa88a89340 => generated 0 bytes in 183 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:02:30.857534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:30.860611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:30.867453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.867948 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.868415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.869464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:30.870234 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:30.870858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:30.907545 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0a01e226-6539-4859-8ee0-19b4f9d81ade tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055271 Sep 18 01:02:30.910508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 305/608] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:02:30 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:31.532207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:31.532907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:31.539065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:31.540038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:31.540471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:31.541157 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:31.541606 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:31.542027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:31.572660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:31.573140 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:31.573574 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:31.574021 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lazy-loading 'fault' on Instance uuid 46b96d5f-1930-4b82-afcb-62726307e170 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:31.614663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:31.622931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f32a1173-0eca-4673-befa-57847323582c tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 200 len: 1312 microversion: 2.1 time: 0.095447 Sep 18 01:02:31.623496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 304/609] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:31 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 1312 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:31.920422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:31.922696 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:31.929022 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:31.929493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:31.929923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:31.930642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:31.931089 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:31.931514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:31.960214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c4e1c44f-f27e-47c0-b96c-7943a674ddf9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043712 Sep 18 01:02:31.960752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 306/610] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:02:31 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:32.645939 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:32.646573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: 46b96d5f-1930-4b82-afcb-62726307e170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:32.652703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.653182 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.653612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.654298 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.654786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.655209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.668073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] HTTP exception thrown: Instance 46b96d5f-1930-4b82-afcb-62726307e170 could not be found. Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Request method failure captured: Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 HTTP/1.1 Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzhkxRTrCUQQ4CHYpp5ib4E3J8zJRLpaPqCNiJzmchcAu_ipeHd4RwYdO1mlchW4uUJ4qXEZ8cWy5sMB5AwukkrGJE4mZR_1pkUaRSmGbYTXpM7QzG3OSBhI3tMi7AEO_3YHBoAPk5T6bb6rWvo2L-te49vgOqSNbUeL_AMq01aAlA Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:02:32.669692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_df01daf86854497fad1d1d016c269ef1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}] Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 54d4e437036640b58fda503c4d213f5c Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 46b96d5f-1930-4b82-afcb-62726307e170 could not be found. Sep 18 01:02:32.671282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '46b96d5f-1930-4b82-afcb-62726307e170'} Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 46b96d5f-1930-4b82-afcb-62726307e170 could not be found. Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:32.672116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 46b96d5f-1930-4b82-afcb-62726307e170 could not be found. Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 46b96d5f-1930-4b82-afcb-62726307e170 could not be found. Sep 18 01:02:32.673498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:32.674786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Returning 404 to user: Instance 46b96d5f-1930-4b82-afcb-62726307e170 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:32.674786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-94addd64-d922-4f39-ae3a-ff50435a87af tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170" status: 404 len: 111 microversion: 2.1 time: 0.028997 Sep 18 01:02:32.674786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 305/611] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:32 2022] GET /compute/v2.1/servers/46b96d5f-1930-4b82-afcb-62726307e170 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:32.679523 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:32.680504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:32.686352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.686930 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.687399 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.688078 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.688527 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.688944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.715745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:02:32.788082 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-428e9cc4-cc8e-451f-a1ae-d18a80e91520 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 204 len: 0 microversion: 2.1 time: 0.111998 Sep 18 01:02:32.788887 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 307/612] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:02:32 2022] DELETE /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:02:32.796776 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:32.797386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:32.803589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.804071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.804506 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.805685 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.806137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.806583 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.936015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:32.943728 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-fc39f370-ff2a-4fff-b4f4-6099c36d2099 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.150837 Sep 18 01:02:32.944287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 306/613] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:32 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:32.973753 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:32.975998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:32.983065 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.983545 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.984002 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.984686 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:32.985140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:32.985565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:32.989302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=4,nova_cell1:UPDATE=4 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:33.017277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5502eb03-8d57-4382-9576-c794c283a53f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047844 Sep 18 01:02:33.017838 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 308/614] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:02:32 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:02:33.085213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:33.085706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:33.091817 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:33.092304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:33.092743 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:33.093431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:33.093881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:33.094308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:33.113349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:33.122207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:02:33.198076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c8db8db5-b5d7-4904-8ffc-92e20364efce tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 204 len: 0 microversion: 2.1 time: 0.116453 Sep 18 01:02:33.198707 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 307/615] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:02:33 2022] DELETE /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:02:33.207129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:33.207756 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:33.213808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:33.214291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:33.214853 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:33.215568 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:33.216038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:33.216450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:33.345903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:33.354245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-77ee04ee-c28c-44cf-93d7-f8cf31979dd0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1679 microversion: 2.1 time: 0.151282 Sep 18 01:02:33.354832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 309/616] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:33 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1679 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:33.967736 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:33.968739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:33.975145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:33.975711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:33.976126 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:33.976747 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:33.977220 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:33.977639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:34.114621 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:34.122877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6e982a04-bbf6-4e0f-b7eb-57ae069a802d tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.159526 Sep 18 01:02:34.123438 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 308/617] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:33 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:34.377654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:34.378271 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:34.385605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:34.386076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:34.386521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:34.387202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:34.387646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:34.388061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:34.542851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:34.550715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-67ebabf2-4ac1-45ad-ad15-fca95371d8fa tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1679 microversion: 2.1 time: 0.176898 Sep 18 01:02:34.551283 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 310/618] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:34 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1679 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:35.146485 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:35.147099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:35.153349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:35.153825 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:35.154256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:35.154972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:35.155416 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:35.155832 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:35.283132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:35.290922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-300069fa-350b-47ea-9770-b66c20f71db9 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.148400 Sep 18 01:02:35.291476 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 309/619] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:35 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:35.573243 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:35.573832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:35.581002 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:35.581494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:35.581929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:35.582672 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:35.583127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:35.583551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:35.718719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:35.726480 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1ad617b0-a4b4-43fe-aad5-6603f7389834 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1679 microversion: 2.1 time: 0.157030 Sep 18 01:02:35.727725 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 311/620] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:35 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1679 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:36.313883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:36.314512 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:36.321589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:36.322178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:36.322645 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:36.323345 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:36.323795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:36.324214 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:36.461694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:36.469986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-abf04672-64fb-469a-bfbf-994b3f774a50 tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.159834 Sep 18 01:02:36.470477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 310/621] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:36 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:36.749975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:36.750656 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:36.757228 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:36.757726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:36.758157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:36.758878 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:36.759335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:36.759753 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:36.893959 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:36.901958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2fbd733d-ebe7-4fc5-8db5-99d4ff6a93dd tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1679 microversion: 2.1 time: 0.156092 Sep 18 01:02:36.902540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 312/622] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:36 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1679 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:37.494126 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:37.494913 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:37.502183 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:37.502718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:37.503153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:37.503855 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:37.504304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:37.504725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:37.658315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:37.667352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-661f6f78-dcda-47d3-9fb7-d456a0ce5b7a tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1522 microversion: 2.1 time: 0.178174 Sep 18 01:02:37.667924 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 311/623] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:37 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:02:37.926108 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:37.926784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:37.933908 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:37.934398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:37.934865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:37.935578 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:37.936024 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:37.936440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:38.164285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:38.172936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9f783b32-83f1-4906-8262-7f6ffa273bda tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1679 microversion: 2.1 time: 0.250693 Sep 18 01:02:38.173905 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 313/624] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:37 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1679 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:38.558735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-87d06512-e759-4cbb-b7ab-8b51a27bd5af req-77da6492-f5eb-444d-844e-1616cbe67e8b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae00028e-df3f-4c20-84a1-e0e14f4cc457", "name": "network-vif-deleted", "tag": "ba2d5bc2-7b64-4dff-96fe-a30ba8d32b9f"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:38.571080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-87d06512-e759-4cbb-b7ab-8b51a27bd5af req-77da6492-f5eb-444d-844e-1616cbe67e8b service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:38.571684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-87d06512-e759-4cbb-b7ab-8b51a27bd5af req-77da6492-f5eb-444d-844e-1616cbe67e8b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:38.572116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-87d06512-e759-4cbb-b7ab-8b51a27bd5af req-77da6492-f5eb-444d-844e-1616cbe67e8b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:38.584588 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-87d06512-e759-4cbb-b7ab-8b51a27bd5af req-77da6492-f5eb-444d-844e-1616cbe67e8b service nova] Creating event network-vif-deleted:ba2d5bc2-7b64-4dff-96fe-a30ba8d32b9f for instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 on hv-cinder-maste Sep 18 01:02:38.591497 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-87d06512-e759-4cbb-b7ab-8b51a27bd5af req-77da6492-f5eb-444d-844e-1616cbe67e8b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037361 Sep 18 01:02:38.592133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 312/625] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:02:38.690569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:38.691492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:38.697992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:38.698504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:38.698939 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:38.699834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:38.700289 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:38.700718 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:38.746041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:38.746862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:38.747650 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:38.748347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lazy-loading 'fault' on Instance uuid ae00028e-df3f-4c20-84a1-e0e14f4cc457 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:38.828276 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '54d4e437036640b58fda503c4d213f5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df01daf86854497fad1d1d016c269ef1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:38.837418 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ca541577-934e-4be6-adce-0dd6258bff0e tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 200 len: 1312 microversion: 2.1 time: 0.150820 Sep 18 01:02:38.845899 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 314/626] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:38 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 1312 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:39.215498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:39.216334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:39.217038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-eeb0756a-7fec-4f3c-abcf-58ddda5f9325 req-90fb8a53-0bb1-48e9-b176-1fc6b652b2fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6c8469ab-e2ff-405e-9da7-e4e93e64fc72", "name": "network-vif-deleted", "tag": "f0d60caf-e51b-4704-af82-ab86c2340a70"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:39.224335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:39.225114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:39.225535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:39.226855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:39.227403 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:39.227908 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:39.231006 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-eeb0756a-7fec-4f3c-abcf-58ddda5f9325 req-90fb8a53-0bb1-48e9-b176-1fc6b652b2fd service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:39.231660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-eeb0756a-7fec-4f3c-abcf-58ddda5f9325 req-90fb8a53-0bb1-48e9-b176-1fc6b652b2fd service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:39.232176 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-eeb0756a-7fec-4f3c-abcf-58ddda5f9325 req-90fb8a53-0bb1-48e9-b176-1fc6b652b2fd service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:39.248221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-eeb0756a-7fec-4f3c-abcf-58ddda5f9325 req-90fb8a53-0bb1-48e9-b176-1fc6b652b2fd service nova] Creating event network-vif-deleted:f0d60caf-e51b-4704-af82-ab86c2340a70 for instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 on hv-cinder-maste Sep 18 01:02:39.255747 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-eeb0756a-7fec-4f3c-abcf-58ddda5f9325 req-90fb8a53-0bb1-48e9-b176-1fc6b652b2fd service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.110877 Sep 18 01:02:39.256909 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 313/627] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:39.316700 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:39.326749 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-831aadc6-f25b-42d4-ac4d-bd03bb185cde tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 200 len: 1611 microversion: 2.1 time: 0.115775 Sep 18 01:02:39.332805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 315/628] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:39 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 1611 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:02:39.861643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:39.862552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] [instance: ae00028e-df3f-4c20-84a1-e0e14f4cc457] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:39.869765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:39.870251 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:39.870795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:39.871510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:39.871965 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:39.872391 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:39.887092 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] HTTP exception thrown: Instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 could not be found. Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Request method failure captured: Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 HTTP/1.1 Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzhkxRTrCUQQ4CHYpp5ib4E3J8zJRLpaPqCNiJzmchcAu_ipeHd4RwYdO1mlchW4uUJ4qXEZ8cWy5sMB5AwukkrGJE4mZR_1pkUaRSmGbYTXpM7QzG3OSBhI3tMi7AEO_3YHBoAPk5T6bb6rWvo2L-te49vgOqSNbUeL_AMq01aAlA Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:02:39.888808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/df01daf86854497fad1d1d016c269ef1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_df01daf86854497fad1d1d016c269ef1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/df01daf86854497fad1d1d016c269ef1"}]}] Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: df01daf86854497fad1d1d016c269ef1 Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-214604638 Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 54d4e437036640b58fda503c4d213f5c Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-ServerRescueNegativeTestJSON-214604638-project Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 could not be found. Sep 18 01:02:39.890273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'ae00028e-df3f-4c20-84a1-e0e14f4cc457'} Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 could not be found. Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:39.891213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 could not be found. Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 could not be found. Sep 18 01:02:39.892704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:39.894202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] Returning 404 to user: Instance ae00028e-df3f-4c20-84a1-e0e14f4cc457 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:39.894202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ea050d25-04a0-44e4-ae93-739e6ef39bdb tempest-ServerRescueNegativeTestJSON-214604638 tempest-ServerRescueNegativeTestJSON-214604638-project] 10.222.0.42 "GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457" status: 404 len: 111 microversion: 2.1 time: 0.032932 Sep 18 01:02:39.894439 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 314/629] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:39 2022] GET /compute/v2.1/servers/ae00028e-df3f-4c20-84a1-e0e14f4cc457 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:40.361785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:40.362518 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:40.370202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:40.371111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:40.371755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:40.372490 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:40.373463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:40.374082 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:40.391599 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 HTTP/1.1 Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:02:40.393296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:02:40.394800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '6c8469ab-e2ff-405e-9da7-e4e93e64fc72'} Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:40.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:02:40.397012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:40.398084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:40.398084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7426b1ba-d045-47d3-bfda-340db2920ee7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 404 len: 111 microversion: 2.1 time: 0.037136 Sep 18 01:02:40.398084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 316/630] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:40 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:40.404037 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cd2b7edd-0435-4b65-affd-74b14bdc204e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '.version_select at 0x7fcab1af24c0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:40.425138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cd2b7edd-0435-4b65-affd-74b14bdc204e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-227418299" status: 202 len: 0 microversion: 2.1 time: 0.024821 Sep 18 01:02:40.425851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 315/631] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 18 01:02:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-227418299 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:02:41.078165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7f487809-8559-409c-a914-8e3d7547624e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: .version_select at 0x7fcab1cdcee0>, body: {"keypair": {"name": "tempest-keypair-727572029", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEt3QtcSFHWfgk5VCWoTw6trTtexK36mzT3Aj6VwYiijSoeEL/Zs3GMaJxVzNe4cwfw/sWJpWVjq5AavBqG/fAOstso53QX0LLoAL43GwUc8fFWT5BrrPmhlFs96Tdr3Ig=="}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:41.095811 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-7f487809-8559-409c-a914-8e3d7547624e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:02:41.100352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-7f487809-8559-409c-a914-8e3d7547624e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:02:41.125239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7f487809-8559-409c-a914-8e3d7547624e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.050947 Sep 18 01:02:41.125848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 317/632] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 01:02:41 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:42.467275 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-635136669", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3227bd19-4c4c-45c0-aee5-e73b3870bae1"}], "security_groups": [{"name": "tempest-securitygroup--971198318"}], "key_name": "tempest-keypair-727572029", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:42.625187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] validate_networks() for [('3227bd19-4c4c-45c0-aee5-e73b3870bae1', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:02:43.261213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:02:43.261648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:02:43.262116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:02:43.262557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:02:43.263176 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:02:43.272365 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:43.272852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:43.273416 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:43.274836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:43.275300 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:43.275915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:43.302901 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:02:43.312987 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Getting quotas for user 2f9751adf8d74ab59b6b0a5d00221405 and project cf1375f9f28c43fda090a236635c4188. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:02:43.327334 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:02:43.330123 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:02:43.399667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-226c6889-fb7b-48ce-a00a-2827f4f837f0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.936665 Sep 18 01:02:43.400723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 316/633] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:02:42 2022] POST /compute/v2.1/servers => generated 401 bytes in 940 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:02:43.412547 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1f073779-5360-4381-bff5-2d5bdc38f9ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:43.413176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1f073779-5360-4381-bff5-2d5bdc38f9ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:43.494851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1f073779-5360-4381-bff5-2d5bdc38f9ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:43.504502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1f073779-5360-4381-bff5-2d5bdc38f9ea tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1240 microversion: 2.1 time: 0.096209 Sep 18 01:02:43.505214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 318/634] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:43 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1240 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:02:44.392768 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-773960957"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:44.393309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:44.399487 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:44.399974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:44.400413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:44.401115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:44.401569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:44.401996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:44.528740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:44.529632 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:44.536044 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:44.536530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:44.536962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:44.537662 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:44.538109 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:44.538549 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:44.561168 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.network.security_group_api [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Removing security group fd6ca814-4786-4459-8a91-d5dda7d3826e from port d02eceef-bf2a-4aaa-b246-5fafd4b75314 Sep 18 01:02:44.631947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:44.640259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3f6f66fd-a56f-4c72-8e6b-e788cf988bb0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1292 microversion: 2.1 time: 0.115800 Sep 18 01:02:44.640770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 319/635] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:44 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1292 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:02:44.959317 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2c8e188e-8c6d-4cb0-840e-6e07bb5921c6 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Action: 'create', calling method: .version_select at 0x7fcab1bde8b0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1283679989", "metadata": {"Type": "work"}}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2c8e188e-8c6d-4cb0-840e-6e07bb5921c6 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Request method failure captured: Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 122 Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm40iUFWPgWBx9GmYZOyT4QGUqC8KSqcxMJYVI7VkdzxSYJHVrSpJ_JrJT7DVBN-PNByW613rQ3R2JJFEXa-yN1lIMEbz66ARLdeiR01VH_TG_0qT2jbxgPuG3Yz_hIQXNXH_Tif-cw9UCPHXXVG9VQ80VgJosVbzRe_xYa9ngEaSIQ Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:02:44.971408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}] Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 151d35e52e6b4a3e83607cc70c2b396c Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1283679989", "metadata": {"Type": "work"}}} Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: Sep 18 01:02:44.973001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1283679989', 'metadata': {'Type': 'work'}}}} Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise error Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi '#$%' Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:44.978985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 18 01:02:44.980579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 18 01:02:44.981987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 18 01:02:44.981987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.981987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2c8e188e-8c6d-4cb0-840e-6e07bb5921c6 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:44.981987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2c8e188e-8c6d-4cb0-840e-6e07bb5921c6 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.172828 Sep 18 01:02:44.981987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 320/636] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 18 01:02:44 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 174 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:02:44.985049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0d737f35-07b6-4b4e-a04c-1d922d15b876 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Action: 'create', calling method: .version_select at 0x7fcab1ce6940>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1407890449", "metadata": {"Type": "work"}}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0d737f35-07b6-4b4e-a04c-1d922d15b876 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Request method failure captured: Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 120 Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm40iUFWPgWBx9GmYZOyT4QGUqC8KSqcxMJYVI7VkdzxSYJHVrSpJ_JrJT7DVBN-PNByW613rQ3R2JJFEXa-yN1lIMEbz66ARLdeiR01VH_TG_0qT2jbxgPuG3Yz_hIQXNXH_Tif-cw9UCPHXXVG9VQ80VgJosVbzRe_xYa9ngEaSIQ Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:02:44.989057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}] Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 151d35e52e6b4a3e83607cc70c2b396c Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1407890449", "metadata": {"Type": "work"}}} Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: Sep 18 01:02:44.990685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1407890449', 'metadata': {'Type': 'work'}}}} Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise error Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi '0' Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:44.991624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 18 01:02:44.993232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 18 01:02:44.994419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 18 01:02:44.994419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:44.994419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0d737f35-07b6-4b4e-a04c-1d922d15b876 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:44.994419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0d737f35-07b6-4b4e-a04c-1d922d15b876 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.009627 Sep 18 01:02:44.994419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 321/637] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 18 01:02:44 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:02:45.005784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-21634367-725e-4743-b5a3-cb8bc23c1920 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Action: 'create', calling method: .version_select at 0x7fcab1ce6f70>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-128264118", "metadata": {"Type": "work"}}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:45.008675 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d13bd7ab-f94e-40b8-9cac-4edfcd7162e8 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/action" status: 202 len: 0 microversion: 2.1 time: 0.619760 Sep 18 01:02:45.009296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 317/638] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:02:44 2022] POST /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/action => generated 0 bytes in 622 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-21634367-725e-4743-b5a3-cb8bc23c1920 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Request method failure captured: Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 118 Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm40iUFWPgWBx9GmYZOyT4QGUqC8KSqcxMJYVI7VkdzxSYJHVrSpJ_JrJT7DVBN-PNByW613rQ3R2JJFEXa-yN1lIMEbz66ARLdeiR01VH_TG_0qT2jbxgPuG3Yz_hIQXNXH_Tif-cw9UCPHXXVG9VQ80VgJosVbzRe_xYa9ngEaSIQ Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:02:45.010202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}] Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 151d35e52e6b4a3e83607cc70c2b396c Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-128264118", "metadata": {"Type": "work"}}} Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: Sep 18 01:02:45.011499 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-128264118', 'metadata': {'Type': 'work'}}}} Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise error Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi '' Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:45.012392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 18 01:02:45.013710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 18 01:02:45.015207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 18 01:02:45.015207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.015207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-21634367-725e-4743-b5a3-cb8bc23c1920 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:45.015207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-21634367-725e-4743-b5a3-cb8bc23c1920 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010064 Sep 18 01:02:45.015207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 322/639] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 18 01:02:44 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 18 01:02:45.022364 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-21aec2f7-d2ba-4021-89ba-0c9890a2c233 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Calling method '.version_select at 0x7fcab1adb5e0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:45.034017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-5ac85be4-a286-4861-90eb-b398c0ff1714 req-7d696334-3f7f-4ce4-aa68-1d7da4faa90d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35d3affc-6943-43c5-9921-109f6fe429fe", "tag": "d02eceef-bf2a-4aaa-b246-5fafd4b75314"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:45.046557 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5ac85be4-a286-4861-90eb-b398c0ff1714 req-7d696334-3f7f-4ce4-aa68-1d7da4faa90d service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:45.047069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5ac85be4-a286-4861-90eb-b398c0ff1714 req-7d696334-3f7f-4ce4-aa68-1d7da4faa90d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:45.047510 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5ac85be4-a286-4861-90eb-b398c0ff1714 req-7d696334-3f7f-4ce4-aa68-1d7da4faa90d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:45.059966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-5ac85be4-a286-4861-90eb-b398c0ff1714 req-7d696334-3f7f-4ce4-aa68-1d7da4faa90d service nova] Creating event network-changed:d02eceef-bf2a-4aaa-b246-5fafd4b75314 for instance 35d3affc-6943-43c5-9921-109f6fe429fe on hv-cinder-maste Sep 18 01:02:45.066686 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-5ac85be4-a286-4861-90eb-b398c0ff1714 req-7d696334-3f7f-4ce4-aa68-1d7da4faa90d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036350 Sep 18 01:02:45.067543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 323/640] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:45.081521 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-21aec2f7-d2ba-4021-89ba-0c9890a2c233 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] HTTP exception thrown: Volume tempest-invalid-1899354940 could not be found. Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-21aec2f7-d2ba-4021-89ba-0c9890a2c233 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Request method failure captured: Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1899354940 HTTP/1.1 Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm40iUFWPgWBx9GmYZOyT4QGUqC8KSqcxMJYVI7VkdzxSYJHVrSpJ_JrJT7DVBN-PNByW613rQ3R2JJFEXa-yN1lIMEbz66ARLdeiR01VH_TG_0qT2jbxgPuG3Yz_hIQXNXH_Tif-cw9UCPHXXVG9VQ80VgJosVbzRe_xYa9ngEaSIQ Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:02:45.083644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}] Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 151d35e52e6b4a3e83607cc70c2b396c Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Volume tempest-invalid-1899354940 could not be found. Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'tempest-invalid-1899354940'} Sep 18 01:02:45.085017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1899354940 could not be found. (HTTP 404) (Request-ID: req-1b8d4c7e-412b-494c-b1dc-51853f44c9d1) Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.085801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 01:02:45.089327 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1899354940 could not be found. Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:45.092664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1899354940 could not be found. Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1899354940 could not be found. Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-21aec2f7-d2ba-4021-89ba-0c9890a2c233 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Returning 404 to user: Volume tempest-invalid-1899354940 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-21aec2f7-d2ba-4021-89ba-0c9890a2c233 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1899354940" status: 404 len: 99 microversion: 2.1 time: 0.067173 Sep 18 01:02:45.095896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 318/641] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 18 01:02:45 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1899354940 => generated 99 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 18 01:02:45.099597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1d00df99-4110-48ef-b59c-3dd323193c08 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.007413 Sep 18 01:02:45.102441 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 324/642] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Sep 18 01:02:45 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 11 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 18 01:02:45.113365 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b49e03c7-5dea-4e8b-b2c6-a7df25247cfc tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005606 Sep 18 01:02:45.114321 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 319/643] 10.222.0.42 () {58 vars in 1137 bytes} [Sun Sep 18 01:02:45 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 8 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 18 01:02:45.123831 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1f169aad-7040-4198-a63c-ae3b9ab28afd tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Calling method '.version_select at 0x7fcab1dc8310>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:45.168515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-1f169aad-7040-4198-a63c-ae3b9ab28afd tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] HTTP exception thrown: Volume e6775d4f-1c02-4e1c-a61a-4358ace052a3 could not be found. Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1f169aad-7040-4198-a63c-ae3b9ab28afd tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Request method failure captured: Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/os-volumes/e6775d4f-1c02-4e1c-a61a-4358ace052a3 HTTP/1.1 Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm40iUFWPgWBx9GmYZOyT4QGUqC8KSqcxMJYVI7VkdzxSYJHVrSpJ_JrJT7DVBN-PNByW613rQ3R2JJFEXa-yN1lIMEbz66ARLdeiR01VH_TG_0qT2jbxgPuG3Yz_hIQXNXH_Tif-cw9UCPHXXVG9VQ80VgJosVbzRe_xYa9ngEaSIQ Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:02:45.173937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}] Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 151d35e52e6b4a3e83607cc70c2b396c Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Volume e6775d4f-1c02-4e1c-a61a-4358ace052a3 could not be found. Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': 'e6775d4f-1c02-4e1c-a61a-4358ace052a3'} Sep 18 01:02:45.176334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e6775d4f-1c02-4e1c-a61a-4358ace052a3 could not be found. (HTTP 404) (Request-ID: req-164e33b7-4665-4bfe-8f77-96a231481c00) Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.177270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 01:02:45.178476 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e6775d4f-1c02-4e1c-a61a-4358ace052a3 could not be found. Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 18 01:02:45.179637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e6775d4f-1c02-4e1c-a61a-4358ace052a3 could not be found. Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e6775d4f-1c02-4e1c-a61a-4358ace052a3 could not be found. Sep 18 01:02:45.181334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.182702 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1f169aad-7040-4198-a63c-ae3b9ab28afd tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Returning 404 to user: Volume e6775d4f-1c02-4e1c-a61a-4358ace052a3 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:45.183452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1f169aad-7040-4198-a63c-ae3b9ab28afd tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/e6775d4f-1c02-4e1c-a61a-4358ace052a3" status: 404 len: 109 microversion: 2.1 time: 0.063555 Sep 18 01:02:45.184248 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 325/644] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:02:45 2022] DELETE /compute/v2.1/os-volumes/e6775d4f-1c02-4e1c-a61a-4358ace052a3 => generated 109 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:45.193443 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8846781d-c860-4260-805d-68abbc513f55 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Calling method '.version_select at 0x7fcab1b8b040>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:45.225496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-8846781d-c860-4260-805d-68abbc513f55 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] HTTP exception thrown: Volume 4c0b681f-a797-45b8-8ac9-bfef2345b33a could not be found. Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8846781d-c860-4260-805d-68abbc513f55 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Request method failure captured: Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/os-volumes/4c0b681f-a797-45b8-8ac9-bfef2345b33a HTTP/1.1 Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm40iUFWPgWBx9GmYZOyT4QGUqC8KSqcxMJYVI7VkdzxSYJHVrSpJ_JrJT7DVBN-PNByW613rQ3R2JJFEXa-yN1lIMEbz66ARLdeiR01VH_TG_0qT2jbxgPuG3Yz_hIQXNXH_Tif-cw9UCPHXXVG9VQ80VgJosVbzRe_xYa9ngEaSIQ Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:02:45.227974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_250267bfa41e4620bf154ec96fcf64ae"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/250267bfa41e4620bf154ec96fcf64ae"}]}] Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 250267bfa41e4620bf154ec96fcf64ae Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesNegativeTest-2076348557 Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 151d35e52e6b4a3e83607cc70c2b396c Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesNegativeTest-2076348557-project Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Volume 4c0b681f-a797-45b8-8ac9-bfef2345b33a could not be found. Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '4c0b681f-a797-45b8-8ac9-bfef2345b33a'} Sep 18 01:02:45.229260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4c0b681f-a797-45b8-8ac9-bfef2345b33a could not be found. (HTTP 404) (Request-ID: req-a723f676-a6a4-4bb0-a48b-37caad93aaa8) Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.230194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:02:45.231470 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4c0b681f-a797-45b8-8ac9-bfef2345b33a could not be found. Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 18 01:02:45.232791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4c0b681f-a797-45b8-8ac9-bfef2345b33a could not be found. Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4c0b681f-a797-45b8-8ac9-bfef2345b33a could not be found. Sep 18 01:02:45.233897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:45.236521 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8846781d-c860-4260-805d-68abbc513f55 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] Returning 404 to user: Volume 4c0b681f-a797-45b8-8ac9-bfef2345b33a could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:45.237708 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8846781d-c860-4260-805d-68abbc513f55 tempest-VolumesNegativeTest-2076348557 tempest-VolumesNegativeTest-2076348557-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/4c0b681f-a797-45b8-8ac9-bfef2345b33a" status: 404 len: 109 microversion: 2.1 time: 0.048315 Sep 18 01:02:45.238825 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 320/645] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:02:45 2022] GET /compute/v2.1/os-volumes/4c0b681f-a797-45b8-8ac9-bfef2345b33a => generated 109 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:45.670331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:45.671109 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:45.679046 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:45.680391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:45.680998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:45.681938 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:45.684808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:45.685477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:45.804550 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:45.812429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0506e639-28c3-43df-9bb5-92cf984903e5 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1298 microversion: 2.1 time: 0.148521 Sep 18 01:02:45.813463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 326/646] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:45 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1298 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:46.399059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:46.399568 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:46.411101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=82,nova_api:UPDATE=2,nova_api:DELETE=2,nova_api:INSERT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:46.436165 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:46.436673 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:46.437119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:46.444283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:46.446530 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:46.449463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:46.562881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-37a8751b-d34c-4c42-a5cb-5e7c9390408d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/ips" status: 200 len: 153 microversion: 2.20 time: 0.426951 Sep 18 01:02:46.564204 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 321/647] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Sep 18 01:02:46 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/ips => generated 153 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 01:02:46.837848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:46.838493 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:46.842306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=78,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:46.856053 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:46.856629 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:46.859685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:46.861620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:46.862341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:46.865750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:46.866292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:46.868626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:46.869160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:46.869280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:46.869652 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:46.870373 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:46.870856 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:46.871339 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:47.056109 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lazy-loading 'services' on Instance uuid 35d3affc-6943-43c5-9921-109f6fe429fe {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:47.185029 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-30bd7a44-98e4-4191-83b1-781a42cdca1a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "DELETE /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments/502c5bc8-cddb-4feb-9f92-837885ed54b7" status: 202 len: 0 microversion: 2.1 time: 0.334136 Sep 18 01:02:47.186836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 322/648] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:02:46 2022] DELETE /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments/502c5bc8-cddb-4feb-9f92-837885ed54b7 => generated 0 bytes in 337 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:02:47.194065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-3d3d2846-062c-4996-bfb8-414fc98f0c52 req-740a6438-9bb2-40d4-a2ee-98df2336811c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35d3affc-6943-43c5-9921-109f6fe429fe", "tag": "d02eceef-bf2a-4aaa-b246-5fafd4b75314"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:47.209124 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-3d3d2846-062c-4996-bfb8-414fc98f0c52 req-740a6438-9bb2-40d4-a2ee-98df2336811c service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:47.209925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-3d3d2846-062c-4996-bfb8-414fc98f0c52 req-740a6438-9bb2-40d4-a2ee-98df2336811c service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:47.211293 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-3d3d2846-062c-4996-bfb8-414fc98f0c52 req-740a6438-9bb2-40d4-a2ee-98df2336811c service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:47.229187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-3d3d2846-062c-4996-bfb8-414fc98f0c52 req-740a6438-9bb2-40d4-a2ee-98df2336811c service nova] Creating event network-changed:d02eceef-bf2a-4aaa-b246-5fafd4b75314 for instance 35d3affc-6943-43c5-9921-109f6fe429fe on hv-cinder-maste Sep 18 01:02:47.235943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-3d3d2846-062c-4996-bfb8-414fc98f0c52 req-740a6438-9bb2-40d4-a2ee-98df2336811c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046604 Sep 18 01:02:47.241932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 323/649] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:47.249253 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:47.251523 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:47.258982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:47.259530 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:47.259971 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:47.260663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:47.261283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:47.261879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:47.275067 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:47.291210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b4d9bea7-ae6a-452e-99a9-4678830447b6 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1365 microversion: 2.1 time: 0.451256 Sep 18 01:02:47.293040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 327/650] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:46 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1365 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:47.345218 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-03756072-8fb8-487d-9826-b479869065b1 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.100463 Sep 18 01:02:47.345849 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 324/651] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:02:47 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments => generated 197 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:47.482856 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-eec49109-9c8e-4399-8620-36d913285473 req-f913eaba-d340-444a-99aa-677c86060c92 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aeb33bda-3ec0-41d5-b1fd-edcabb801279", "tag": "31d344bf-7a01-4651-9813-cae9ac176973"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:47.500570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-eec49109-9c8e-4399-8620-36d913285473 req-f913eaba-d340-444a-99aa-677c86060c92 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:47.501058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-eec49109-9c8e-4399-8620-36d913285473 req-f913eaba-d340-444a-99aa-677c86060c92 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:47.502589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-eec49109-9c8e-4399-8620-36d913285473 req-f913eaba-d340-444a-99aa-677c86060c92 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:47.529124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-eec49109-9c8e-4399-8620-36d913285473 req-f913eaba-d340-444a-99aa-677c86060c92 service nova] Creating event network-changed:31d344bf-7a01-4651-9813-cae9ac176973 for instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 on hv-cinder-maste Sep 18 01:02:47.534363 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-eec49109-9c8e-4399-8620-36d913285473 req-f913eaba-d340-444a-99aa-677c86060c92 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055328 Sep 18 01:02:47.535608 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 328/652] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:48.322449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:48.323880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:48.338040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.338993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.341186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.342266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.343531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.345986 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.362746 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:48.367088 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:48.382900 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.385687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.387450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.389763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.391698 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.392551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.467235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5b546598-3c86-4968-9437-cdbd426b01f7 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.112938 Sep 18 01:02:48.467897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 329/653] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:02:48 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments => generated 25 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:02:48.484122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:48.484881 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:48.493327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.494052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.494698 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.495585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.496202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.496623 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.626660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] HTTP exception thrown: Instance 35d3affc-6943-43c5-9921-109f6fe429fe is not attached to volume 502c5bc8-cddb-4feb-9f92-837885ed54b7 Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Request method failure captured: Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments/502c5bc8-cddb-4feb-9f92-837885ed54b7 HTTP/1.1 Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm2I2b7x4bRKkbU2tvryzW1PBvezt6nRAU9pEcphtFUDc9bVURuTfWObhl1Zn0mkmuC8iODleRmyhlpmKilLg7ckd6DIjPF3qva6Stm_qpQKQ-yS3pX7DGkVOAKzr5B8RyZYXJJxFtgm3gA8U3-z0UKMRJSRQxTIPS-I7da_vBDMuH8 Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: c9f737f6350f4d1f993708b1f90a862e Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-TestMinimumBasicScenario-929481336 Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:02:48.629273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c9f737f6350f4d1f993708b1f90a862e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c9f737f6350f4d1f993708b1f90a862e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c9f737f6350f4d1f993708b1f90a862e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c9f737f6350f4d1f993708b1f90a862e"}]}] Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-TestMinimumBasicScenario-929481336 Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: c9f737f6350f4d1f993708b1f90a862e Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-TestMinimumBasicScenario-929481336 Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-TestMinimumBasicScenario-929481336-project Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 20a50c6db67f4d818a6fbeaafce299d9 Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-TestMinimumBasicScenario-929481336-project Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:02:48.630731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 35d3affc-6943-43c5-9921-109f6fe429fe is not attached to volume 502c5bc8-cddb-4feb-9f92-837885ed54b7 Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'server_id': '35d3affc-6943-43c5-9921-109f6fe429fe', 'id': '502c5bc8-cddb-4feb-9f92-837885ed54b7'} Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 502c5bc8-cddb-4feb-9f92-837885ed54b7. Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:48.631425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35d3affc-6943-43c5-9921-109f6fe429fe is not attached to volume 502c5bc8-cddb-4feb-9f92-837885ed54b7 Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35d3affc-6943-43c5-9921-109f6fe429fe is not attached to volume 502c5bc8-cddb-4feb-9f92-837885ed54b7 Sep 18 01:02:48.632548 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:02:48.634392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Returning 404 to user: Instance 35d3affc-6943-43c5-9921-109f6fe429fe is not attached to volume 502c5bc8-cddb-4feb-9f92-837885ed54b7 {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:48.635480 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f8be93b5-8544-4ceb-9922-26362d720672 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "DELETE /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments/502c5bc8-cddb-4feb-9f92-837885ed54b7" status: 404 len: 154 microversion: 2.1 time: 0.155635 Sep 18 01:02:48.636217 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 330/654] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:02:48 2022] DELETE /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe/os-volume_attachments/502c5bc8-cddb-4feb-9f92-837885ed54b7 => generated 154 bytes in 157 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:48.641296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:48.642161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:48.650654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.651828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.652477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.653386 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.654272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.654914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.685696 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lazy-loading 'flavor' on Instance uuid 8a40bb3d-0090-4ce2-9782-c61f0161e064 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:48.707838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:48.724706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4de24039-3c27-41b0-938c-2db403ec735f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1365 microversion: 2.1 time: 0.409717 Sep 18 01:02:48.725816 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 325/655] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:48 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1365 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:48.772829 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:48.936745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7855e027-9b8c-44fd-99b3-6b8630149cc5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/action" status: 202 len: 0 microversion: 2.20 time: 0.299048 Sep 18 01:02:48.937407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 331/656] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 18 01:02:48 2022] POST /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/action => generated 0 bytes in 300 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:02:48.946205 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:48.947138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:48.955521 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.956347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.957097 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:48.957881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:48.958563 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:48.958984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:49.116651 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:49.127541 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:49.129558 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:49.130772 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-041c8471-8e00-4b71-b8e9-7d501be3c973 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.189054 Sep 18 01:02:49.146481 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 326/657] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:48 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:02:49.749726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:49.750435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:49.756753 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:49.757268 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:49.757753 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:49.758946 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:49.759385 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:49.759812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:49.841221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:49.841995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:49.848581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:49.849066 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:49.849676 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:49.850371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:49.850848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:49.851437 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:49.877524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:02:49.965109 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d7283c89-9ff6-4827-ab22-19e6f053251d tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "DELETE /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 204 len: 0 microversion: 2.1 time: 0.127556 Sep 18 01:02:49.966066 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 327/658] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:02:49 2022] DELETE /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 18 01:02:49.975331 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:49.976132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:49.982953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:49.983609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:49.984233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:49.984931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:49.985534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:49.986105 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:49.986370 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:49.991661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:50.000997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e2f797d8-7f31-4aa9-8c7e-b9eaad416dca tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1365 microversion: 2.1 time: 0.255894 Sep 18 01:02:50.002312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 332/659] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:49 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1365 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:50.149753 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:50.157841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-83ca621d-85d4-4c5e-a519-dbd226620461 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1553 microversion: 2.1 time: 0.187208 Sep 18 01:02:50.158349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:50.159028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:50.165140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:50.165634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:50.166049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:50.166759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:50.167204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:50.167897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:50.173271 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 328/660] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:49 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1553 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:50.320271 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:50.329551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:50.331285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:50.332435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-19fdc06f-1116-48b7-a1b8-4b3100bd8a99 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.179219 Sep 18 01:02:50.333002 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 333/661] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:50 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:02:51.024972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:51.025573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:51.031911 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:51.032575 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:51.033005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:51.033689 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:51.034153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:51.034583 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:51.181196 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:51.181814 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:51.188179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:51.197286 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:51.198050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:51.198233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7e0a1d21-c511-4e5b-bbf4-33a2375496be tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1365 microversion: 2.1 time: 0.177073 Sep 18 01:02:51.198846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 329/662] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:51 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1365 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:51.199172 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:51.199917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:51.200380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:51.200805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:51.333349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:51.342889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-de9e4f0a-a105-4260-9b7c-658c835e5b50 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1553 microversion: 2.1 time: 0.165226 Sep 18 01:02:51.343473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 334/663] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:51 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1553 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:02:51.357220 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:51.357861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:51.363688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:51.364163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:51.364593 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:51.365273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:51.365718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:51.366133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:51.509084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:51.517460 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:51.519018 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:51.520191 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a5c9a5c0-93a6-42df-ada1-cda507da21e9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.166863 Sep 18 01:02:51.520992 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 330/664] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:51 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:02:51.984962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-1989b963-1a61-498f-852f-9d8aa104ae70 req-2913d693-5d55-4d3f-a3e4-b673b5749e33 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aeb33bda-3ec0-41d5-b1fd-edcabb801279", "name": "network-vif-plugged", "status": "completed", "tag": "31d344bf-7a01-4651-9813-cae9ac176973"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:51.998297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-1989b963-1a61-498f-852f-9d8aa104ae70 req-2913d693-5d55-4d3f-a3e4-b673b5749e33 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:51.998810 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-1989b963-1a61-498f-852f-9d8aa104ae70 req-2913d693-5d55-4d3f-a3e4-b673b5749e33 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:51.999416 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-1989b963-1a61-498f-852f-9d8aa104ae70 req-2913d693-5d55-4d3f-a3e4-b673b5749e33 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:52.013327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-1989b963-1a61-498f-852f-9d8aa104ae70 req-2913d693-5d55-4d3f-a3e4-b673b5749e33 service nova] Creating event network-vif-plugged:31d344bf-7a01-4651-9813-cae9ac176973 for instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 on hv-cinder-maste Sep 18 01:02:52.018183 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-1989b963-1a61-498f-852f-9d8aa104ae70 req-2913d693-5d55-4d3f-a3e4-b673b5749e33 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037382 Sep 18 01:02:52.018830 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 335/665] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:52.222041 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:52.222837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:52.230663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:52.231224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:52.231807 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:52.232775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:52.233232 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:52.233874 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:52.367316 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:52.368009 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:52.374436 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:52.374975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:52.375408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:52.376873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:52.377495 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:52.378070 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:52.379465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:52.387873 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0a526cc8-9ee3-4b45-85d6-cc7dbeb50d1c tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1365 microversion: 2.1 time: 0.169649 Sep 18 01:02:52.388534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 331/666] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:52 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1365 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:52.545968 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:52.548172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:52.555054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:52.555867 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:52.557478 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:52.558890 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:52.559571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:52.560063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:52.619017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:52.637149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-015f4464-0fec-4263-808e-1d0e2128ea12 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1553 microversion: 2.1 time: 0.274894 Sep 18 01:02:52.637992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 336/667] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:52 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1553 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:52.829528 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:52.840587 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:52.842130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:52.843518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d38bb2e0-864a-4d0d-8697-415a83c30511 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.302018 Sep 18 01:02:52.844153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 332/668] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:52 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:02:53.081485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-c362dc70-cf22-4747-b7c2-f7e432007969 req-7b8a8f31-70a8-44dc-9ed5-2907a931729e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35d3affc-6943-43c5-9921-109f6fe429fe", "name": "network-vif-deleted", "tag": "d02eceef-bf2a-4aaa-b246-5fafd4b75314"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:53.091105 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-c362dc70-cf22-4747-b7c2-f7e432007969 req-7b8a8f31-70a8-44dc-9ed5-2907a931729e service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:53.091614 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-c362dc70-cf22-4747-b7c2-f7e432007969 req-7b8a8f31-70a8-44dc-9ed5-2907a931729e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:53.092186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-c362dc70-cf22-4747-b7c2-f7e432007969 req-7b8a8f31-70a8-44dc-9ed5-2907a931729e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:53.103538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-c362dc70-cf22-4747-b7c2-f7e432007969 req-7b8a8f31-70a8-44dc-9ed5-2907a931729e service nova] Creating event network-vif-deleted:d02eceef-bf2a-4aaa-b246-5fafd4b75314 for instance 35d3affc-6943-43c5-9921-109f6fe429fe on hv-cinder-maste Sep 18 01:02:53.108399 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-c362dc70-cf22-4747-b7c2-f7e432007969 req-7b8a8f31-70a8-44dc-9ed5-2907a931729e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030833 Sep 18 01:02:53.108987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 337/669] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:53.412751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:53.413348 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:53.416245 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:53.416903 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=6 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:53.423630 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:53.424252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:53.424693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:53.425399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:53.425840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:53.426264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:53.563918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:53.571807 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-88ac2283-5c92-4be7-a1c0-17d4b71cef9b tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1365 microversion: 2.1 time: 0.162906 Sep 18 01:02:53.572375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 333/670] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:53 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1365 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:53.660294 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:02:53.665043 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:53.665739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:53.672180 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:53.672661 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:53.673095 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:53.673779 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:53.674232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:53.674685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:53.739027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:53.746791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-37bf539f-692c-43ad-811b-457b19d0100c tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1342 microversion: 2.1 time: 0.087912 Sep 18 01:02:53.747600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 338/671] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:53 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1342 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:53.868643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:53.869331 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:53.875479 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:53.876145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:53.876863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:53.877812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:53.878276 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:53.878729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.025917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:54.034765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:54.036226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:54.037448 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f2922a0d-ae3f-4b5e-893d-5a803e8d2a1f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.172936 Sep 18 01:02:54.038011 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 334/672] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:53 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:02:54.595591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:54.596193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:54.602724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:54.603197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:54.603627 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.604312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:54.604749 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:54.605205 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.732079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:54.739708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b215072a-9630-4870-b187-3c908e960e62 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1553 microversion: 2.1 time: 0.148029 Sep 18 01:02:54.740349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 339/673] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:54 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1553 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:54.762613 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:54.763152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:54.768958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:54.769525 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:54.769634 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:54.770013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:54.770444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.771157 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:54.771608 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:54.772070 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.775618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:54.776160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:54.776611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.777327 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:54.777781 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:54.778204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.813222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:54.813708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:54.814170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:54.814650 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lazy-loading 'fault' on Instance uuid 35d3affc-6943-43c5-9921-109f6fe429fe {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:02:54.898120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '20a50c6db67f4d818a6fbeaafce299d9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c9f737f6350f4d1f993708b1f90a862e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:54.907130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-311aafde-08ec-4298-aebc-4b438f9edac0 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 200 len: 1347 microversion: 2.1 time: 0.141915 Sep 18 01:02:54.907879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 340/674] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:54 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 1347 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:02:54.934733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d4e87dd7-9126-499f-930e-cb0cd2c7d1b3 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.175187 Sep 18 01:02:54.935394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 335/675] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 01:02:54 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-interface => generated 284 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:55.065331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:55.066048 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:55.072576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:55.073439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:55.073576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:55.074216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:55.074691 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:55.075117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:55.381782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:55.389790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:55.393045 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:55.393558 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-54a420be-81ea-4270-be85-41ea173ab91b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.333167 Sep 18 01:02:55.394331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 341/676] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:55 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:02:55.932173 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:55.932770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] [instance: 35d3affc-6943-43c5-9921-109f6fe429fe] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:55.938970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:55.939458 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:55.939888 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:55.940594 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:55.941045 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:55.941466 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:55.955763 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] HTTP exception thrown: Instance 35d3affc-6943-43c5-9921-109f6fe429fe could not be found. Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Request method failure captured: Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe HTTP/1.1 Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm2I2b7x4bRKkbU2tvryzW1PBvezt6nRAU9pEcphtFUDc9bVURuTfWObhl1Zn0mkmuC8iODleRmyhlpmKilLg7ckd6DIjPF3qva6Stm_qpQKQ-yS3pX7DGkVOAKzr5B8RyZYXJJxFtgm3gA8U3-z0UKMRJSRQxTIPS-I7da_vBDMuH8 Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: c9f737f6350f4d1f993708b1f90a862e Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-TestMinimumBasicScenario-929481336 Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:02:55.957729 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c9f737f6350f4d1f993708b1f90a862e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c9f737f6350f4d1f993708b1f90a862e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c9f737f6350f4d1f993708b1f90a862e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c9f737f6350f4d1f993708b1f90a862e"}]}] Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-TestMinimumBasicScenario-929481336 Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: c9f737f6350f4d1f993708b1f90a862e Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-TestMinimumBasicScenario-929481336 Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-TestMinimumBasicScenario-929481336-project Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 20a50c6db67f4d818a6fbeaafce299d9 Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-TestMinimumBasicScenario-929481336-project Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 35d3affc-6943-43c5-9921-109f6fe429fe could not be found. Sep 18 01:02:55.959379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '35d3affc-6943-43c5-9921-109f6fe429fe'} Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35d3affc-6943-43c5-9921-109f6fe429fe could not be found. Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:02:55.960180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35d3affc-6943-43c5-9921-109f6fe429fe could not be found. Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35d3affc-6943-43c5-9921-109f6fe429fe could not be found. Sep 18 01:02:55.961929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:02:55.963080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Returning 404 to user: Instance 35d3affc-6943-43c5-9921-109f6fe429fe could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:02:55.963080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-05ce7048-5966-44e9-a73f-dd37fb94524a tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe" status: 404 len: 111 microversion: 2.1 time: 0.031374 Sep 18 01:02:55.963080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 336/677] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:02:55 2022] GET /compute/v2.1/servers/35d3affc-6943-43c5-9921-109f6fe429fe => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:02:55.968315 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cc3c429e-0ad1-4c3f-9890-869b1d7bc122 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] Calling method '.version_select at 0x7fcab1b348b0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:55.988398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cc3c429e-0ad1-4c3f-9890-869b1d7bc122 tempest-TestMinimumBasicScenario-929481336 tempest-TestMinimumBasicScenario-929481336-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-108321933" status: 202 len: 0 microversion: 2.1 time: 0.023251 Sep 18 01:02:55.988398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 342/678] 10.222.0.42 () {58 vars in 1269 bytes} [Sun Sep 18 01:02:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-108321933 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:02:56.311287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-e05324f0-180b-4f2d-a334-08e5c5de13cf req-9959c038-4857-4c55-ba41-45b342b0d731 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aeb33bda-3ec0-41d5-b1fd-edcabb801279", "tag": "31d344bf-7a01-4651-9813-cae9ac176973"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:02:56.321480 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e05324f0-180b-4f2d-a334-08e5c5de13cf req-9959c038-4857-4c55-ba41-45b342b0d731 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:56.321960 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e05324f0-180b-4f2d-a334-08e5c5de13cf req-9959c038-4857-4c55-ba41-45b342b0d731 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:56.322449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e05324f0-180b-4f2d-a334-08e5c5de13cf req-9959c038-4857-4c55-ba41-45b342b0d731 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:56.335669 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-e05324f0-180b-4f2d-a334-08e5c5de13cf req-9959c038-4857-4c55-ba41-45b342b0d731 service nova] Creating event network-changed:31d344bf-7a01-4651-9813-cae9ac176973 for instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 on hv-cinder-maste Sep 18 01:02:56.341008 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-e05324f0-180b-4f2d-a334-08e5c5de13cf req-9959c038-4857-4c55-ba41-45b342b0d731 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033739 Sep 18 01:02:56.341761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 337/679] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:02:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:02:56.421496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:56.422117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:56.440072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:56.441032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:56.441635 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:56.443058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:56.443987 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:56.444579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:56.756330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:56.767738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:56.772085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:56.773287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1ec63c94-2434-48d8-830c-5eec0cebc2b1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.355874 Sep 18 01:02:56.790664 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 343/680] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:56 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:02:57.812825 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:57.813667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:57.820412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:57.821129 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:57.821758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:57.822597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:57.823152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:57.823688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:58.018942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:58.028309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:58.029801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:58.031242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0290d795-cbfb-4cbe-b03c-5ccf7bcd5428 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.223735 Sep 18 01:02:58.032082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 338/681] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:57 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:02:59.059388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:02:59.059989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:02:59.068346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:59.068722 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:59.069149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:59.069842 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:02:59.070447 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:02:59.071998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:02:59.229916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:59.240363 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:59.242332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:02:59.243740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3943062f-5d5f-4077-b110-4f2d93692d60 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.189283 Sep 18 01:02:59.244490 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 344/682] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:02:59 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:00.283337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:00.284035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:00.293209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:00.293815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:00.294147 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:00.294884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:00.295342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:00.295769 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:00.478683 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:00.497120 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:00.499742 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:00.501191 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ef60465c-0668-4948-9463-aac82d445dc9 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.234386 Sep 18 01:03:00.515284 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 339/683] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:00 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:01.528867 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:01.529485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:01.536347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:01.536832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:01.537271 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:01.537978 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:01.538432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:01.538890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:01.677608 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:01.685859 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:01.687334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:01.688492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b71d2d1a-d68e-4026-842c-011c471391cf tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1745 microversion: 2.20 time: 0.163833 Sep 18 01:03:01.689057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 345/684] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:01 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1745 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:02.715171 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:02.715777 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:02.722033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:02.722543 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:02.723026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:02.723666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:02.724122 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:02.724556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:02.877264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:02.888442 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:02.891069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:02.892585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-33f40791-6870-421d-abe9-7481812d6b20 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1745 microversion: 2.20 time: 0.181612 Sep 18 01:03:02.893954 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 340/685] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:02 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1745 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:03.921017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:03.921565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:03.931766 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:03.932266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:03.932703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:03.933423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:03.933877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:03.934300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:04.231933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:04.243524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:04.245213 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:04.246503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a5a9a487-86b5-4199-aee9-14af0ca6d60f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.329292 Sep 18 01:03:04.247524 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 346/686] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:03 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:05.299884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:05.300499 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:05.320130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:05.323433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:05.324216 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:05.325067 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:05.325637 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:05.329812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:05.537141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:05.548456 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:05.549915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:05.551373 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3c57715a-4c66-465c-ba7f-3cfab063cb18 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.275528 Sep 18 01:03:05.552427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 341/687] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:05 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:06.418586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-b2e4da88-d150-4b7a-8b20-41e4e4c8bffc req-3e849dfe-8aaf-4157-b4f6-9a5e4fabdddd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a40bb3d-0090-4ce2-9782-c61f0161e064", "name": "network-vif-plugged", "status": "completed", "tag": "5025d940-e81a-4c19-8fda-e4fac2888b2a"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:06.432324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b2e4da88-d150-4b7a-8b20-41e4e4c8bffc req-3e849dfe-8aaf-4157-b4f6-9a5e4fabdddd service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:06.432827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b2e4da88-d150-4b7a-8b20-41e4e4c8bffc req-3e849dfe-8aaf-4157-b4f6-9a5e4fabdddd service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:06.433449 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b2e4da88-d150-4b7a-8b20-41e4e4c8bffc req-3e849dfe-8aaf-4157-b4f6-9a5e4fabdddd service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:06.447785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-b2e4da88-d150-4b7a-8b20-41e4e4c8bffc req-3e849dfe-8aaf-4157-b4f6-9a5e4fabdddd service nova] Creating event network-vif-plugged:5025d940-e81a-4c19-8fda-e4fac2888b2a for instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 on hv-cinder-maste Sep 18 01:03:06.453167 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-b2e4da88-d150-4b7a-8b20-41e4e4c8bffc req-3e849dfe-8aaf-4157-b4f6-9a5e4fabdddd service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.268092 Sep 18 01:03:06.454077 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 347/688] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:03:06.584849 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:06.587258 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:06.595436 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:06.595912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:06.598943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:06.599713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:06.600873 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:06.601535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:06.793998 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:06.804132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:06.806364 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:06.807720 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-fea76364-bb84-4ecd-b731-65a9c99a3cd8 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.228506 Sep 18 01:03:06.808462 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 342/689] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:06 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:07.838098 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:07.840616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:07.855032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:07.855523 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:07.855951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:07.856652 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:07.857101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:07.857519 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:08.019491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:08.028540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:08.030032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:08.031330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e23cd9ba-d433-4d70-b547-21b81e2ac628 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.201841 Sep 18 01:03:08.032465 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 348/690] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:07 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:09.064356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:09.065684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:09.081627 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:09.082402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:09.083022 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:09.083922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:09.085634 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:09.087066 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:09.264276 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:09.274626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:09.279963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:09.279963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-95cbcd4c-bf8a-4894-b19d-cc77e9daab38 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.223894 Sep 18 01:03:09.279963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 343/691] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:09 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:10.304242 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:10.304933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:10.315532 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:10.316417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:10.316828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:10.317553 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:10.318043 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:10.318640 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:10.537701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:10.546024 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:10.547761 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:10.549132 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-67919ed3-393d-4547-ba54-e21083cff81d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.249096 Sep 18 01:03:10.566188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 349/692] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:10 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:11.576143 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:11.576860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:11.583878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:11.584621 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:11.585385 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:11.586662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:11.588645 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:11.589256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:11.747507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:11.766245 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:11.767810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:11.768984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b1184881-5cf9-40e6-9494-04e7793a7909 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.197283 Sep 18 01:03:11.769542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 344/693] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:11 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:12.796107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:12.796679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:12.803630 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:12.804298 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:12.804913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:12.805784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:12.806380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:12.806956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:12.965111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:12.969187 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:12.975791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:12.977432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:12.978694 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8baa95aa-4b66-4baa-a5c0-a3586a2d4f7f tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.187099 Sep 18 01:03:12.979302 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 350/694] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:12 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:03:14.003465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:14.004096 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:14.015248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:14.015714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:14.016150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:14.016847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:14.017298 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:14.017720 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:14.159875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:14.169802 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:14.171300 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:14.172469 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8fa23103-46c0-4b28-bc6b-f470b0ff94a6 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1740 microversion: 2.20 time: 0.172854 Sep 18 01:03:14.173059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 345/695] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:13 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1740 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:15.198262 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:15.198857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:15.205419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:15.205893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:15.206325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:15.207308 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:15.207774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:15.208194 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:15.351540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:15.360370 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:15.361811 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:15.363188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f54f5a85-ea70-4095-805e-5ad0f6819901 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1722 microversion: 2.20 time: 0.168814 Sep 18 01:03:15.363816 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 351/696] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:15 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1722 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:16.389023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:16.389659 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:16.396039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:16.396474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:16.396908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:16.397601 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:16.398057 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:16.398495 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:16.417973 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=100 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:16.533732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:16.542035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:16.543683 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:16.544824 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0a7af1ea-366a-40ab-881f-61caa8268371 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1722 microversion: 2.20 time: 0.159616 Sep 18 01:03:16.545371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 346/697] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:16 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:03:17.569166 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:17.569755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:17.572904 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=53,nova_api:DELETE=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:17.578188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:17.578709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:17.579145 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:17.580015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:17.580819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:17.581810 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:17.717070 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:17.725383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:17.727373 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:17.729286 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8cfdca46-de4d-47cb-b748-e5f4359d9fde tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1665 microversion: 2.20 time: 0.163899 Sep 18 01:03:17.729868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 352/698] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:17 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1665 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:17.754292 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cf090023-f78a-4984-b723-7d588bc61ad2"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:17.757411 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:17.759771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:17.765081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:17.765571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:17.766035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:17.766778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:17.767232 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:17.767655 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:18.062920 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e704ddc8-3c6f-4c21-b2a7-0b8c22602e8d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.312410 Sep 18 01:03:18.063805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 347/699] 10.222.0.42 () {62 vars in 1364 bytes} [Sun Sep 18 01:03:17 2022] POST /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-volume_attachments => generated 188 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 18 01:03:18.191740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:18.192237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:18.198919 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:18.199401 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:18.199836 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:18.200565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:18.201017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:18.201437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:18.474880 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b5eeb24f-660d-4820-9d07-6d874b0d1f3a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-volume_attachments/cf090023-f78a-4984-b723-7d588bc61ad2" status: 202 len: 0 microversion: 2.20 time: 0.286836 Sep 18 01:03:18.475596 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 353/700] 10.222.0.42 () {60 vars in 1457 bytes} [Sun Sep 18 01:03:18 2022] DELETE /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-volume_attachments/cf090023-f78a-4984-b723-7d588bc61ad2 => generated 0 bytes in 288 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:03:18.546735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:18.547326 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:18.554285 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:18.554807 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:18.555252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:18.556606 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:18.557090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:18.557549 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:18.586610 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120831) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 18 01:03:18.660055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-eb287d0a-ec33-4642-8c1b-ef0ff8a01ee2 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "POST /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/action" status: 202 len: 0 microversion: 2.20 time: 0.117474 Sep 18 01:03:18.660662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 348/701] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 18 01:03:18 2022] POST /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/action => generated 0 bytes in 119 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 18 01:03:18.669214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:18.669819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:18.676956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:18.677441 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:18.677899 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:18.678632 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:18.679087 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:18.679606 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:18.844382 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:18.853147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:18.854670 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:18.855865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6efebb07-b13c-4978-84b3-de3075048676 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1673 microversion: 2.20 time: 0.190989 Sep 18 01:03:18.856433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 354/702] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:18 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1673 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:19.598072 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1001709604", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:19.649379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] validate_networks() for None {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:03:19.881267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:19.881870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:19.888815 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:19.889299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:19.889749 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:19.890448 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:19.890918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:19.891360 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:19.955929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:03:19.956351 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:03:19.957083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:03:19.957497 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:03:19.957905 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:03:19.967536 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:19.968013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:19.968442 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:19.969160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:19.969637 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:19.970056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:19.989747 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Getting quotas for project 54761854b3e9496ea28cc4c8224c25bf. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:03:19.993878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Getting quotas for user 7d5aef87230d4424a73e87248ae81d78 and project 54761854b3e9496ea28cc4c8224c25bf. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:03:20.004681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:03:20.008588 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:03:20.069300 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f370e016-5295-4dfd-b8d7-13833a172a95 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.475193 Sep 18 01:03:20.070469 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 349/703] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:03:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 477 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:03:20.084143 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d55b297a-eec0-44b5-8751-649415d53306 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:20.086689 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d55b297a-eec0-44b5-8751-649415d53306 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:20.157190 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:20.168821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:20.171912 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:20.173640 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d7d21e0b-3187-4c16-959f-0100818d35a3 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.295926 Sep 18 01:03:20.186484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 355/704] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:19 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 18 01:03:20.186843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d55b297a-eec0-44b5-8751-649415d53306 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:20.194884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d55b297a-eec0-44b5-8751-649415d53306 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1215 microversion: 2.1 time: 0.113515 Sep 18 01:03:20.195483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 350/705] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:20 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1215 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:20.208862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-f1d37a29-dfd3-43c0-bcab-efe3eed27d78 req-ac2b92ce-b5ce-4d98-b95e-156b7d0a976d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a40bb3d-0090-4ce2-9782-c61f0161e064", "name": "network-vif-unplugged", "status": "completed", "tag": "5025d940-e81a-4c19-8fda-e4fac2888b2a"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:20.221657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-f1d37a29-dfd3-43c0-bcab-efe3eed27d78 req-ac2b92ce-b5ce-4d98-b95e-156b7d0a976d service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:20.222141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-f1d37a29-dfd3-43c0-bcab-efe3eed27d78 req-ac2b92ce-b5ce-4d98-b95e-156b7d0a976d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:20.222618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-f1d37a29-dfd3-43c0-bcab-efe3eed27d78 req-ac2b92ce-b5ce-4d98-b95e-156b7d0a976d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:20.237393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-f1d37a29-dfd3-43c0-bcab-efe3eed27d78 req-ac2b92ce-b5ce-4d98-b95e-156b7d0a976d service nova] Creating event network-vif-unplugged:5025d940-e81a-4c19-8fda-e4fac2888b2a for instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 on hv-cinder-maste Sep 18 01:03:20.246734 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-f1d37a29-dfd3-43c0-bcab-efe3eed27d78 req-ac2b92ce-b5ce-4d98-b95e-156b7d0a976d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.070797 Sep 18 01:03:20.255491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 356/706] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:21.203699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:21.204311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:21.211774 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.212894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.213433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.214380 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.219089 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.219597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.233309 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:21.234049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:21.240391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.240880 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.241310 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.242197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.242845 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.243272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.334433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:21.343403 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-63f90009-f0bb-4f38-90fc-5c9a8805d90b tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1267 microversion: 2.1 time: 0.127537 Sep 18 01:03:21.343984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 357/707] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:21 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1267 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:21.393776 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:21.407870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:21.409342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:21.412040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:21.412655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:21.415345 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2b362a52-258d-4427-a482-4d4a886b4fa7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.210760 Sep 18 01:03:21.421173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.422057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.422788 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.424019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.424707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.425376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.433161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 351/708] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:21 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:21.462284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:03:21.607269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-64c5ad7b-29ed-4c04-bd5b-ddc3ce5d44ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "DELETE /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 204 len: 0 microversion: 2.1 time: 0.198477 Sep 18 01:03:21.611817 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 358/709] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:03:21 2022] DELETE /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:03:21.627402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:21.628644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:21.638789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.639362 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.639982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.641219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:21.642089 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:21.642877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:21.878118 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:21.887572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-aa7f3270-e0c5-4f6e-8283-b2a1b9aa7656 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.273795 Sep 18 01:03:21.899087 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 352/710] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:21 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:22.368979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:22.370567 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:22.379705 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:22.380489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:22.381093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:22.382110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:22.382799 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:22.383451 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:22.440829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:22.441623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:22.448323 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:22.449111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:22.449827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:22.450700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:22.451423 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:22.452059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:22.458337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:22.475220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-335f9596-1654-4b2a-8d89-9fc3bc81b4c2 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.108934 Sep 18 01:03:22.486497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 359/711] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:22 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:22.616424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:22.625179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:22.626709 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:22.627880 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8b5792e0-afa0-4300-8e47-37681a1984e5 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.192145 Sep 18 01:03:22.628636 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 353/712] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:22 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:22.912487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:22.913124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:22.919773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:22.920250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:22.920680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:22.921383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:22.921832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:22.922253 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:23.062138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:23.070812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9aa3ad72-a195-4943-a335-87235d3e2099 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.162214 Sep 18 01:03:23.071380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 360/713] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:22 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:23.503127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:23.504451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:23.510884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:23.511699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:23.512390 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:23.513217 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:23.513775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:23.514410 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:23.583618 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:23.591950 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f3bf63ab-4815-4419-92db-d40cc6906ede tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.092606 Sep 18 01:03:23.592509 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 354/714] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:23 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:23.654143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:23.654793 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:23.661439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:23.661928 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:23.662360 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:23.663736 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:23.664213 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:23.664639 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:23.793033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:23.801019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:23.802452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:23.803618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e290d151-ad8c-4f4a-9116-688ac2a42dae tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.154068 Sep 18 01:03:23.804178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 361/715] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:23 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:24.095641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:24.096304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:24.102282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:24.102835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:24.103270 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:24.103952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:24.104408 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:24.104815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:24.249455 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:24.258658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1a9d3ab2-b932-4d2d-b34f-aa598b6f5ce5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.167595 Sep 18 01:03:24.259719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 355/716] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:24 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:24.614604 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:24.615209 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:24.622046 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:24.622549 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:24.622997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:24.623710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:24.624182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:24.624620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:24.689352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:24.697287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-15de0e0d-99ea-4012-89ec-0cde4e482ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.086693 Sep 18 01:03:24.697930 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 362/717] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:24 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:24.828660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:24.829266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:24.836048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:24.836553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:24.837096 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:24.838345 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:24.838820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:24.839241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:24.981351 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:24.989617 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:24.991127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:24.992291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cf6acbda-c881-40ca-9836-c68b4d0600f7 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.167660 Sep 18 01:03:24.993025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 356/718] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:24 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:25.284505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:25.285080 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:25.291536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:25.292028 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:25.292598 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:25.293319 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:25.293795 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:25.294196 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:25.425275 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:25.433969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-06850255-af8a-4e83-8bd9-514a10a92afa tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.153495 Sep 18 01:03:25.434572 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 363/719] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:25 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:25.721835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:25.722857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:25.729917 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:25.730731 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:25.731569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:25.732543 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:25.733344 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:25.734084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:25.809193 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:25.817842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c26f0f5a-b171-445d-b552-a74b83ff1246 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.100349 Sep 18 01:03:25.825972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 357/720] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:25 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:26.019875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:26.020653 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:26.029603 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:26.030415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:26.030958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:26.031764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:26.032251 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:26.032680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:26.035813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:26.170084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:26.179427 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:26.181619 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:26.182945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0912470b-5af2-4725-ac15-cdd9b6b8a576 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.168848 Sep 18 01:03:26.192999 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 364/721] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:26 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:26.460866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:26.461610 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:26.468852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:26.470032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:26.470484 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:26.471242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:26.471689 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:26.472106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:26.615354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:26.623555 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f88b5374-7101-443d-8c00-79801ce778f7 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.168615 Sep 18 01:03:26.624311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 358/722] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:26 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:26.841928 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:26.842561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:26.849182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:26.849667 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:26.850107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:26.850854 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:26.851314 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:26.851735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:26.920244 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:26.927856 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c29411c2-208c-4452-b3b0-e503f2647ea7 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.090440 Sep 18 01:03:26.928424 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 365/723] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:26 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:27.219436 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:27.219948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:27.226187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:27.226712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:27.227150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:27.227835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:27.228286 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:27.228726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:27.368035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:27.377044 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:27.378572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:27.379746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-79777bb9-dd58-488e-a3f1-40e3ca95ca41 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.164563 Sep 18 01:03:27.380351 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 359/724] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:27 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:27.646318 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:27.647020 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:27.653491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:27.653966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:27.654397 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:27.655126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:27.655603 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:27.656020 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:27.803335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:27.813549 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-82ffbc94-2d10-4636-993e-262a6ef8610b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.171180 Sep 18 01:03:27.814124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 366/725] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:27 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:27.951113 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:27.951726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:27.958818 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:27.959309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:27.959744 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:27.960500 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:27.960960 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:27.961381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:28.062820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:28.072583 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-925ae5b8-2a5e-43fb-af77-cd8a6d7544ee tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.125287 Sep 18 01:03:28.073154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 360/726] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:27 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:28.406155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:28.406761 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:28.413717 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:28.414249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:28.414728 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:28.415453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:28.415905 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:28.416352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:28.564155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:28.577574 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:28.579719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:28.581364 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-69ed98ef-44fd-40b3-9d5c-fc8e41d651fd tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.179312 Sep 18 01:03:28.582955 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 367/727] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:28 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:28.837547 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:28.838149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:28.844269 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:28.844749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:28.845177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:28.845857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:28.846309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:28.846751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:28.850764 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:29.013152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:29.023598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-014dbc7a-4535-4c60-9d61-f382a15db2f9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.189729 Sep 18 01:03:29.036255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 361/728] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:28 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:29.096869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:29.097406 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:29.104389 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:29.104889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:29.105322 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:29.106033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:29.106501 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:29.106924 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:29.189757 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:29.200408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e284a072-5eaa-42fc-87c3-802ea305da04 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.107507 Sep 18 01:03:29.201267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 368/729] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:29 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:29.607027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:29.607636 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:29.613598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:29.614084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:29.614582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:29.615277 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:29.615739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:29.616160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:29.745124 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:29.754357 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:29.755946 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:29.757196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-dd5c1eec-374b-4ece-abf4-27f4ba3e1452 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.154155 Sep 18 01:03:29.768654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 362/730] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:29 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:30.049910 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:30.051470 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:30.058970 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:30.059726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:30.060426 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:30.061436 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:30.062021 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:30.062599 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:30.228897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:30.230591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:30.231206 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:30.233778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:30.234808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=8 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:30.240182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-044bf0fc-bcf3-4937-bf51-bd51ae4a8566 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.194765 Sep 18 01:03:30.240809 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 369/731] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:30 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:30.240974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:30.241499 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:30.242131 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:30.243871 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:30.244330 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:30.244877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:30.328544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:30.337349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3673555a-bcad-43d7-81bf-9731a0f72857 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.110883 Sep 18 01:03:30.338515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 363/732] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:30 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:30.783673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:30.784285 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:30.790266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:30.791500 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:30.791975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:30.792681 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:30.793179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:30.793610 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:30.937467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:30.946055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:30.947625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:30.948800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f72d062f-c367-42fb-8655-5b4c323dedad tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.169118 Sep 18 01:03:30.949361 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 370/733] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:30 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:31.265740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:31.266358 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:31.273952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:31.274498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:31.275693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:31.276386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:31.276837 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:31.277257 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:31.362326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:31.363018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:31.370166 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:31.370750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:31.371201 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:31.371916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:31.372329 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:31.372749 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:31.414011 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:31.422854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ccb273c8-eb95-4d36-841b-ce4cfd5dca04 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.161107 Sep 18 01:03:31.423744 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 364/734] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:31 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:31.445387 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:31.456423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e4146441-20a2-434b-8cbb-e1a28fa4c04e tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1275 microversion: 2.1 time: 0.097732 Sep 18 01:03:31.459012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 371/735] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:31 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1275 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:31.974404 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:31.975136 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:31.981706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:31.982183 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:31.982656 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:31.983335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:31.983784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:31.984204 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:32.112612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:32.121820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:32.123342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:32.124568 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e099446e-d081-45e3-bac3-85301df71e5b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.154050 Sep 18 01:03:32.125116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 365/736] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:31 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:32.448656 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:32.449266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:32.455837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:32.456316 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:32.456744 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:32.457690 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:32.458136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:32.458579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:32.481371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:32.482068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:32.488354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:32.488856 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:32.489323 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:32.490024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:32.490524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:32.490952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:32.573908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:32.581821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6125b5b5-0e5c-43fa-97ca-fcbc68d75491 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1285 microversion: 2.1 time: 0.105112 Sep 18 01:03:32.582687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 366/737] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:32 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1285 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:32.617679 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:32.625472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bbabdd42-f731-481d-8051-7df681c5edb6 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.180770 Sep 18 01:03:32.626155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 372/738] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:32 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:32.848910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:32.849425 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:32.855712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:32.856276 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:32.856710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:32.857441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:32.857892 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:32.858317 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:32.881527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-58cacdb9-29ea-431f-bbb8-ac370948e0a4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/ips" status: 200 len: 148 microversion: 2.1 time: 0.037606 Sep 18 01:03:32.882093 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 367/739] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:03:32 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/ips => generated 148 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:33.154000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:33.155266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:33.165943 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:33.166908 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:33.167515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:33.168543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:33.169113 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:33.169607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:33.312490 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:33.320539 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:33.322467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:33.323602 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-957649ab-1ca5-4b68-afb3-e1efa75ad6c1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.177596 Sep 18 01:03:33.324174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 373/740] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:33 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:33.605361 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:33.605963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:33.612523 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:33.613011 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:33.613441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:33.614127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:33.614619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:33.615082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:33.650442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:33.652622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:33.659135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:33.659647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:33.660129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:33.661256 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:33.661720 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:33.662145 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:33.680889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:33.689225 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-46b90765-b8ce-4df1-aeb3-e84e0da99131 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1273 microversion: 2.1 time: 0.087812 Sep 18 01:03:33.689798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 368/741] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:33 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1273 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:33.882473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:33.891889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-376d5099-d1ed-41de-8c67-4cdb617b6c97 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1575 microversion: 2.1 time: 0.246410 Sep 18 01:03:33.892475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 374/742] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:33 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1575 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:34.224170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "547dc1c7-1950-4765-ba28-dcc0b13cfd2f"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:34.226796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:34.233129 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.233620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.234058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.234807 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.235306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.235741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.355257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:34.355913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:34.364543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.365018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.365460 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.366209 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.366690 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.371071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.599021 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:34.599806 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ae394919-f749-460d-9be1-ce979e77d7ed tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "POST /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.379477 Sep 18 01:03:34.600513 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 369/743] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:03:34 2022] POST /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments => generated 194 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:34.609596 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:34.611165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:34.613438 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e6d3d098-f280-474d-a15f-1f52a82dba2b tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.262393 Sep 18 01:03:34.614566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 375/744] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:34 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:34.714150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:34.714859 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:34.722948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.723686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.724264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.724994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.725443 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.725885 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.916073 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:34.917565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:34.924246 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.924883 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.925547 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.926342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:34.927079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:34.927655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:34.968375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:34.977579 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-af0216f8-5044-4783-bd6f-a4df39b31be5 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.267759 Sep 18 01:03:34.978307 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 370/745] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:34 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:35.051138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:35.060826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1e412283-369c-46a6-a848-7f5924463a85 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1508 microversion: 2.1 time: 0.148597 Sep 18 01:03:35.067979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 376/746] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:34 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1508 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:35.278809 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-91106625-2669-412c-a9da-afb7aec8f899 req-45f72a09-0819-4e53-bc9a-922785c59895 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "443c0c33-d16f-4b9b-86dc-99207ed72962", "name": "network-vif-deleted", "tag": "644cd5b5-288e-4651-9d36-20ed8d528fcb"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:35.292896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-91106625-2669-412c-a9da-afb7aec8f899 req-45f72a09-0819-4e53-bc9a-922785c59895 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:35.293733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-91106625-2669-412c-a9da-afb7aec8f899 req-45f72a09-0819-4e53-bc9a-922785c59895 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:35.294408 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-91106625-2669-412c-a9da-afb7aec8f899 req-45f72a09-0819-4e53-bc9a-922785c59895 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:35.312505 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-91106625-2669-412c-a9da-afb7aec8f899 req-45f72a09-0819-4e53-bc9a-922785c59895 service nova] Creating event network-vif-deleted:644cd5b5-288e-4651-9d36-20ed8d528fcb for instance 443c0c33-d16f-4b9b-86dc-99207ed72962 on hv-cinder-maste Sep 18 01:03:35.339308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-91106625-2669-412c-a9da-afb7aec8f899 req-45f72a09-0819-4e53-bc9a-922785c59895 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064526 Sep 18 01:03:35.340894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 371/747] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:35.562607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-5d39c4cf-48ce-41ed-8ad0-67e1962af0d8 req-e0cdc5e0-fd22-46e8-96da-196d8ac73881 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e28379c-2544-434d-bd34-794bccec8b71", "tag": "e64f91b6-4c08-40d4-9722-1795a059189f"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:35.573344 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5d39c4cf-48ce-41ed-8ad0-67e1962af0d8 req-e0cdc5e0-fd22-46e8-96da-196d8ac73881 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:35.574063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5d39c4cf-48ce-41ed-8ad0-67e1962af0d8 req-e0cdc5e0-fd22-46e8-96da-196d8ac73881 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:35.574708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5d39c4cf-48ce-41ed-8ad0-67e1962af0d8 req-e0cdc5e0-fd22-46e8-96da-196d8ac73881 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:35.587467 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-5d39c4cf-48ce-41ed-8ad0-67e1962af0d8 req-e0cdc5e0-fd22-46e8-96da-196d8ac73881 service nova] Creating event network-changed:e64f91b6-4c08-40d4-9722-1795a059189f for instance 8e28379c-2544-434d-bd34-794bccec8b71 on hv-cinder-maste Sep 18 01:03:35.603701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-5d39c4cf-48ce-41ed-8ad0-67e1962af0d8 req-e0cdc5e0-fd22-46e8-96da-196d8ac73881 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042759 Sep 18 01:03:35.606244 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 377/748] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:03:35.641421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:35.642119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:35.649030 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:35.649765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:35.650515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:35.651920 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:35.652375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:35.652957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:35.806019 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:35.821108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:35.822629 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:35.824329 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d2415992-ae8a-491f-b946-67183fc355f1 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1731 microversion: 2.20 time: 0.187606 Sep 18 01:03:35.824901 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 372/749] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:35 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1731 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:36.003880 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:36.004920 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:36.019375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:36.019862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:36.020292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:36.024939 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:36.026730 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:36.028815 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:36.086125 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:36.089433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:36.104239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:36.105212 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:36.105815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:36.106757 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:36.107427 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:36.107910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:36.206498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:36.207761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:36.209154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:36.209814 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lazy-loading 'fault' on Instance uuid 443c0c33-d16f-4b9b-86dc-99207ed72962 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:03:36.361664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:36.370631 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3aac1bd3-0e8c-4f7b-80f1-0c210c8c2a40 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 200 len: 1228 microversion: 2.1 time: 0.289280 Sep 18 01:03:36.387272 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 373/750] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:36 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 1228 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:36.441827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:36.449957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1f011cdc-53cb-44dd-a886-fa4789fa4dea tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.451587 Sep 18 01:03:36.455601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 378/751] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:35 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:36.853280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:36.853909 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:36.865220 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:36.866255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:36.866791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:36.867578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:36.868105 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:36.868760 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:37.105996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:37.115106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:37.116735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:37.118134 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-62e59400-903c-493b-b115-1f00c1badf0d tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1718 microversion: 2.20 time: 0.268753 Sep 18 01:03:37.118834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 374/752] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:03:36 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1718 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:03:37.399456 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:37.400059 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:37.407165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:37.407647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:37.408079 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:37.408770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:37.409221 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:37.409646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:37.422703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] HTTP exception thrown: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Request method failure captured: Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 HTTP/1.1 Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm2QgJpMeVeVcLZ9xjhZKdL8tWRNuqVlXbl2NMu42aYJIy4vR0JFiqcvkHQFss6ZNrK7UGOrxwMS_HU0diziLoPZijNOM_0W3JRqZf6y9fGBmVrMvx3-uc9S57PXVceGxz_bmsOYR3_dBbtQNiKzG6z3tDNmrbRcFwU54fN8ZFm4jaQ Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:03:37.424330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}] Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 823fa0c3eeeb489591c639b81afc02bc Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:03:37.425944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '443c0c33-d16f-4b9b-86dc-99207ed72962'} Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:37.427039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:03:37.428551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:37.429976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Returning 404 to user: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:37.429976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c19f42b0-7b8c-4226-81db-424729bf0624 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 404 len: 111 microversion: 2.1 time: 0.030163 Sep 18 01:03:37.429976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 379/753] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:37 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:37.472997 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:37.473600 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:37.479447 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:37.479991 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:37.480358 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:37.481039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:37.481487 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:37.481905 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:37.534348 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-442139730", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-58756093", "security_groups": [{"name": "tempest-secgroup-smoke-821199059"}], "block_device_mapping_v2": [{"uuid": "59129af3-cd17-48cc-9f9b-4f1d0af3cdf7", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "4ff6b087-81f6-47ef-8679-6af29cd7449d"}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:37.626925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:37.634408 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0d835ba9-1e41-4fa3-a731-57c31080cfcd tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.165186 Sep 18 01:03:37.634998 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 375/754] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:37 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:37.657051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] validate_networks() for [('4ff6b087-81f6-47ef-8679-6af29cd7449d', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:03:38.208435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:03:38.208435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:03:38.208699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:03:38.209111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:03:38.209528 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:03:38.220425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:38.221247 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:38.221895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:38.222741 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:38.223248 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:38.223719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:38.244992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for project 6958209222b44905bcb4baa1b8c26c58. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:03:38.250189 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for user 823fa0c3eeeb489591c639b81afc02bc and project 6958209222b44905bcb4baa1b8c26c58. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:03:38.293150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-6f276cf3-1abb-4c83-8896-3837ef5dcc9d req-b738c4b2-b063-4ed1-a086-53df3cef5104 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a40bb3d-0090-4ce2-9782-c61f0161e064", "name": "network-vif-plugged", "status": "completed", "tag": "5025d940-e81a-4c19-8fda-e4fac2888b2a"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:38.307348 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-6f276cf3-1abb-4c83-8896-3837ef5dcc9d req-b738c4b2-b063-4ed1-a086-53df3cef5104 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:38.308628 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-6f276cf3-1abb-4c83-8896-3837ef5dcc9d req-b738c4b2-b063-4ed1-a086-53df3cef5104 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:38.309301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-6f276cf3-1abb-4c83-8896-3837ef5dcc9d req-b738c4b2-b063-4ed1-a086-53df3cef5104 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:38.324821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-6f276cf3-1abb-4c83-8896-3837ef5dcc9d req-b738c4b2-b063-4ed1-a086-53df3cef5104 service nova] Creating event network-vif-plugged:5025d940-e81a-4c19-8fda-e4fac2888b2a for instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 on hv-cinder-maste Sep 18 01:03:38.328460 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:03:38.330851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-6f276cf3-1abb-4c83-8896-3837ef5dcc9d req-b738c4b2-b063-4ed1-a086-53df3cef5104 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043809 Sep 18 01:03:38.331688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='59129af3-cd17-48cc-9f9b-4f1d0af3cdf7',volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:03:38.331934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 376/755] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:38.488009 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR oslo.messaging._drivers.impl_rabbit [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [10cfaf4b-5e12-43a4-9fa3-5ee2457cba09] AMQP server on 10.222.0.42:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 18 01:03:38.658001 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:38.658735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:38.665160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:38.665658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:38.666100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:38.667362 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:38.667829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:38.668255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:38.881711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:38.885252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:38.892241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6629e909-a099-477f-8a5c-59dabdfb1ab3 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.238566 Sep 18 01:03:38.893045 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 377/756] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:38 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:39.496798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO oslo.messaging._drivers.impl_rabbit [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [10cfaf4b-5e12-43a4-9fa3-5ee2457cba09] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 33544. Sep 18 01:03:39.500919 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0f041906-54f4-4278-9ff9-17837bc1372c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.976949 Sep 18 01:03:39.501570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 380/757] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:03:37 2022] POST /compute/v2.1/servers => generated 405 bytes in 1978 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:03:39.513269 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a2c950c8-f6bb-415b-a7cf-21b9e12c7021 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:39.513942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a2c950c8-f6bb-415b-a7cf-21b9e12c7021 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:39.568925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a2c950c8-f6bb-415b-a7cf-21b9e12c7021 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:39.575967 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a2c950c8-f6bb-415b-a7cf-21b9e12c7021 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1092 microversion: 2.1 time: 0.067194 Sep 18 01:03:39.576549 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 378/758] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:39 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1092 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:39.916075 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:39.917366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:39.923477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:39.923996 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:39.924429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:39.925119 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:39.925569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:39.925988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:40.059761 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:40.068202 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-484db741-4b28-48b0-be07-c616f33e5034 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.155842 Sep 18 01:03:40.068930 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 381/759] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:39 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:40.598866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:40.599731 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:40.607934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:40.608366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:40.608795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:40.609744 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:40.610190 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:40.610669 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:40.696694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:40.705567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-dae8f0e8-5ea9-44e1-bf1f-c6e6d09399a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1198 microversion: 2.1 time: 0.110777 Sep 18 01:03:40.707856 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 379/760] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:40 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1198 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:41.092622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:41.093259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:41.099685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:41.100412 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:41.100592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:41.101295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:41.101754 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:41.102249 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:41.337099 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:41.340018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=100 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:41.346779 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1692c4b6-1f30-4cb8-87b5-71fd9588867a tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.258574 Sep 18 01:03:41.347359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 382/761] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:41 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:41.733053 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:41.733633 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:41.740261 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:41.740750 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:41.741184 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:41.741879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:41.742326 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:41.742794 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.003886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:42.012417 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b4331025-31d7-4ab6-b245-a59e3b14dd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1275 microversion: 2.1 time: 0.283399 Sep 18 01:03:42.013619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 380/762] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:41 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1275 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:42.353703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-5d6f26f2-73a3-4404-a0a0-8543b1c241c7 req-a7f94b1a-9ef7-4e88-a8eb-56cf2d6d87eb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e28379c-2544-434d-bd34-794bccec8b71", "name": "network-vif-plugged", "status": "completed", "tag": "e64f91b6-4c08-40d4-9722-1795a059189f"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:42.364507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5d6f26f2-73a3-4404-a0a0-8543b1c241c7 req-a7f94b1a-9ef7-4e88-a8eb-56cf2d6d87eb service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.366284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5d6f26f2-73a3-4404-a0a0-8543b1c241c7 req-a7f94b1a-9ef7-4e88-a8eb-56cf2d6d87eb service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.366284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-5d6f26f2-73a3-4404-a0a0-8543b1c241c7 req-a7f94b1a-9ef7-4e88-a8eb-56cf2d6d87eb service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.372449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:42.373424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:42.386106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.388033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.389202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.389970 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-5d6f26f2-73a3-4404-a0a0-8543b1c241c7 req-a7f94b1a-9ef7-4e88-a8eb-56cf2d6d87eb service nova] Creating event network-vif-plugged:e64f91b6-4c08-40d4-9722-1795a059189f for instance 8e28379c-2544-434d-bd34-794bccec8b71 on hv-cinder-maste Sep 18 01:03:42.390257 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.390982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.391474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.402675 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-5d6f26f2-73a3-4404-a0a0-8543b1c241c7 req-a7f94b1a-9ef7-4e88-a8eb-56cf2d6d87eb service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046609 Sep 18 01:03:42.402675 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 383/763] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:42.450944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-e32d1611-8ff6-4454-84bb-b7498df336ff req-5b675dbc-0076-4c6f-83cf-e177f5617e47 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4aaef7df-7526-47f3-a235-e7565d13df51", "tag": "d89c76f8-4e53-4830-b1fe-dff8b48e622b"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:42.493064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-e32d1611-8ff6-4454-84bb-b7498df336ff req-5b675dbc-0076-4c6f-83cf-e177f5617e47 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.493497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-e32d1611-8ff6-4454-84bb-b7498df336ff req-5b675dbc-0076-4c6f-83cf-e177f5617e47 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.493873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-e32d1611-8ff6-4454-84bb-b7498df336ff req-5b675dbc-0076-4c6f-83cf-e177f5617e47 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.535706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-e32d1611-8ff6-4454-84bb-b7498df336ff req-5b675dbc-0076-4c6f-83cf-e177f5617e47 service nova] Creating event network-changed:d89c76f8-4e53-4830-b1fe-dff8b48e622b for instance 4aaef7df-7526-47f3-a235-e7565d13df51 on hv-cinder-maste Sep 18 01:03:42.540336 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-e32d1611-8ff6-4454-84bb-b7498df336ff req-5b675dbc-0076-4c6f-83cf-e177f5617e47 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099905 Sep 18 01:03:42.542197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 384/764] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:42.549824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:42.550586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:42.558642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.559144 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.559573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.560864 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.561151 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.561732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.636945 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:42.644669 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-56bc2964-1794-4623-a373-b06bd9ee07eb tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.276955 Sep 18 01:03:42.645231 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 381/765] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:42 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:42.662953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lazy-loading 'services' on Instance uuid aeb33bda-3ec0-41d5-b1fd-edcabb801279 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:03:42.775336 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-00cca377-b1fb-461b-9b92-7a532dfd9650 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments/547dc1c7-1950-4765-ba28-dcc0b13cfd2f" status: 202 len: 0 microversion: 2.1 time: 0.230589 Sep 18 01:03:42.776300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 385/766] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:03:42 2022] DELETE /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments/547dc1c7-1950-4765-ba28-dcc0b13cfd2f => generated 0 bytes in 233 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:03:42.784937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:42.787118 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:42.793582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.794066 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.794516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.795212 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:42.795661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:42.796079 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:42.825877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c8c255b0-c7af-44a8-8dd4-5eeb1c76c12e tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044958 Sep 18 01:03:42.826424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 382/767] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:03:42 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:43.037050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:43.037432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:43.044354 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:43.044868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:43.045311 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:43.046680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:43.047152 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:43.047585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:43.273915 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:43.282536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fab92336-2cb0-49c0-bd34-b91efe828400 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1275 microversion: 2.1 time: 0.250129 Sep 18 01:03:43.283110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 386/768] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:43 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1275 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:43.669453 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:43.670061 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:43.676138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:43.676660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:43.677164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:43.677847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:43.678297 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:43.678817 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:43.806234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:43.814520 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-bb4bbb8a-608a-4460-850b-9c3e1d1e86b1 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1315 microversion: 2.1 time: 0.148909 Sep 18 01:03:43.815163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 383/769] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:43 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1315 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:43.838585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:43.840923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:43.847254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:43.847732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:43.848158 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:43.848839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:43.849281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:43.849696 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:43.879300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c1d544b9-39a3-466a-aeb6-7f69d5fcd4c4 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044715 Sep 18 01:03:43.879847 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 387/770] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:03:43 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:44.305161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:44.305765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:44.309160 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:44.313100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:44.313579 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:44.314008 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:44.314717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:44.315167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:44.315583 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:44.449165 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:44.457266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3c60ba44-73b5-4753-9608-e7ecb46ea614 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.155921 Sep 18 01:03:44.458154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 384/771] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:44 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:44.837812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:44.838418 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:44.846211 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:44.846920 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:44.847150 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:44.847857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:44.848315 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:44.848740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:44.893970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:44.896951 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:44.905153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:44.905648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:44.906084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:44.906826 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:44.907283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:44.907713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:44.941337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ffe8dece-df82-48df-b3d1-1ec3715acf47 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052793 Sep 18 01:03:44.941918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 385/772] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:03:44 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:03:45.066051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:45.077057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6a0cbd1c-851a-462c-b814-1b1dec789884 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1464 microversion: 2.1 time: 0.243201 Sep 18 01:03:45.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 388/773] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:44 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1464 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:45.125848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:45.126444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:45.132955 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.133553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.134066 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.134912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.136199 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.136508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.166240 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:03:45.270163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2de5de75-92bc-4db2-8b15-e03f2c716cd8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "DELETE /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 204 len: 0 microversion: 2.1 time: 0.147974 Sep 18 01:03:45.271037 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 386/774] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:03:45 2022] DELETE /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:03:45.280339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:45.281047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:45.290566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.292737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.293689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.295169 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.295770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.296276 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.433224 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:45.442201 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2b666bb6-828f-4043-8d04-ec8b2abe9497 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1470 microversion: 2.1 time: 0.166493 Sep 18 01:03:45.442812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 389/775] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:45 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1470 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:45.481721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:45.482401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:45.488309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.488783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.489213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.489888 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.490338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.490786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.628320 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:45.635068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b4869a07-e34d-433a-a286-25ab43093794 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.157965 Sep 18 01:03:45.635740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 387/776] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:45 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:45.954179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:45.956439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:45.963328 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.964293 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.964721 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.965409 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:45.965854 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:45.966270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:45.997442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-96e36d82-0180-4507-909e-77eee31a2451 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047189 Sep 18 01:03:45.998031 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 390/777] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:03:45 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:03:46.065382 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:46.066003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:46.073708 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.074269 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.074723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.075438 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.075887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.076306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.104650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:03:46.183424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0d6fbf82-8ef6-4b5b-b025-0ca15d0b620f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 204 len: 0 microversion: 2.1 time: 0.121675 Sep 18 01:03:46.184012 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 388/778] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:03:46 2022] DELETE /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:03:46.192204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:46.192828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:46.194785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:46.200440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.200914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.201343 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.202024 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.202485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.202934 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.330374 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:46.338318 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b5e2f830-58f3-4614-91e6-c584320b3cd9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1678 microversion: 2.1 time: 0.149961 Sep 18 01:03:46.339519 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 391/779] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:46 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1678 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:46.465772 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:46.466687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:46.474384 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.474977 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.475412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.476266 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.476741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.477840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.625173 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:46.632988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-23dbcfc7-2038-4dd3-86f1-f2218f1bf357 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1470 microversion: 2.1 time: 0.171507 Sep 18 01:03:46.633567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 389/780] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:46 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1470 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:46.657593 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:46.658203 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:46.664556 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.665296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.665778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.666519 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:46.667001 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:46.667425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:46.801212 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:46.809568 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-da47e498-586d-4218-b7ce-23caf28f0fb4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.155884 Sep 18 01:03:46.810131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 392/781] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:46 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:47.363878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:47.364486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:47.371139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:47.371623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:47.372058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:47.372770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:47.373309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:47.373736 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:47.501297 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:47.509496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-12e49031-87f1-4294-b344-9e593efd167a tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1678 microversion: 2.1 time: 0.150573 Sep 18 01:03:47.510271 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 390/782] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:47 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1678 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:47.657075 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:47.657616 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:47.664161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:47.664649 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:47.665094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:47.665779 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:47.666223 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:47.666701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:47.810514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:47.818723 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7f36ac05-1e9f-4498-9531-aa8c028aa5db tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1470 microversion: 2.1 time: 0.165523 Sep 18 01:03:47.819303 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 393/783] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:47 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1470 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:47.832050 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:47.832915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:47.839886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:47.840383 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:47.840820 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:47.841517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:47.841975 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:47.842404 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:47.970195 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:47.977934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-18362bc8-a382-4559-a7dd-255be8fe056e tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.149881 Sep 18 01:03:47.978519 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 391/784] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:47 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:48.548785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:48.549384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:48.551240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:48.552025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:48.560282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:48.560784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:48.561227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:48.561942 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:48.562397 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:48.562847 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:48.769838 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:48.778530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d76d5748-9b99-4643-b30b-d6a764622b34 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1678 microversion: 2.1 time: 0.233544 Sep 18 01:03:48.779278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 394/785] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:48 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1678 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:48.844175 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:48.845402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:48.852448 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:48.853584 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:48.853961 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:48.855438 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:48.856039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:48.856592 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:48.861395 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:49.001147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:49.001764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:49.002640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7d5aef87230d4424a73e87248ae81d78', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '54761854b3e9496ea28cc4c8224c25bf', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:49.015673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:49.016395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:49.016846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:49.017644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:49.018042 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:49.018482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:49.042279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-123dd793-3b94-48b5-9bec-6a44150977b8 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 200 len: 1428 microversion: 2.1 time: 0.200575 Sep 18 01:03:49.046936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 392/786] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:48 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 1428 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:49.052204 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-77b6b926-d8cd-4da6-a452-9a11fe3651b9 req-8cae9486-47d4-4ad7-95af-a2b9400941e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e28379c-2544-434d-bd34-794bccec8b71", "name": "network-vif-deleted", "tag": "e64f91b6-4c08-40d4-9722-1795a059189f"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:49.067891 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-77b6b926-d8cd-4da6-a452-9a11fe3651b9 req-8cae9486-47d4-4ad7-95af-a2b9400941e8 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:49.068788 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-77b6b926-d8cd-4da6-a452-9a11fe3651b9 req-8cae9486-47d4-4ad7-95af-a2b9400941e8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:49.069386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-77b6b926-d8cd-4da6-a452-9a11fe3651b9 req-8cae9486-47d4-4ad7-95af-a2b9400941e8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:49.091916 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-77b6b926-d8cd-4da6-a452-9a11fe3651b9 req-8cae9486-47d4-4ad7-95af-a2b9400941e8 service nova] Creating event network-vif-deleted:e64f91b6-4c08-40d4-9722-1795a059189f for instance 8e28379c-2544-434d-bd34-794bccec8b71 on hv-cinder-maste Sep 18 01:03:49.097723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-77b6b926-d8cd-4da6-a452-9a11fe3651b9 req-8cae9486-47d4-4ad7-95af-a2b9400941e8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049242 Sep 18 01:03:49.098969 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 393/787] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:49.185533 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:49.193716 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-62749288-93ea-47cd-8285-7cbedf34f86b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.196533 Sep 18 01:03:49.194278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 395/788] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:48 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:03:49.732827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:49.737988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-1814d14d-2222-4f07-8184-a988d0272ea1 req-14115c58-460c-40ac-a02b-db24a8c79fe1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aeb33bda-3ec0-41d5-b1fd-edcabb801279", "name": "network-vif-deleted", "tag": "31d344bf-7a01-4651-9813-cae9ac176973"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:49.757370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-1814d14d-2222-4f07-8184-a988d0272ea1 req-14115c58-460c-40ac-a02b-db24a8c79fe1 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:49.757994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-1814d14d-2222-4f07-8184-a988d0272ea1 req-14115c58-460c-40ac-a02b-db24a8c79fe1 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:49.758739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-1814d14d-2222-4f07-8184-a988d0272ea1 req-14115c58-460c-40ac-a02b-db24a8c79fe1 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:49.775921 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-1814d14d-2222-4f07-8184-a988d0272ea1 req-14115c58-460c-40ac-a02b-db24a8c79fe1 service nova] Creating event network-vif-deleted:31d344bf-7a01-4651-9813-cae9ac176973 for instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 on hv-cinder-maste Sep 18 01:03:49.786184 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-1814d14d-2222-4f07-8184-a988d0272ea1 req-14115c58-460c-40ac-a02b-db24a8c79fe1 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054017 Sep 18 01:03:49.787612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 394/789] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:49.812614 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:49.813203 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:49.822080 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:49.822645 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:49.823141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:49.823862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:49.824300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:49.824720 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:49.897484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:49.906032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fbada2bf-2414-4543-918b-f63de439de7f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1325 microversion: 2.1 time: 0.105310 Sep 18 01:03:49.906830 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 396/790] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:49 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1325 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:50.066626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:50.067173 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] [instance: 8e28379c-2544-434d-bd34-794bccec8b71] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:50.073857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:50.074443 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:50.075054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:50.075942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:50.076534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:50.077081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:50.096796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] HTTP exception thrown: Instance 8e28379c-2544-434d-bd34-794bccec8b71 could not be found. Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Request method failure captured: Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 HTTP/1.1 Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm5W70FEmBnfihMq0w9V3BKCf_dKPLeyEJPdgTPxF2UrQP-tggKDZEZuq2CBr6e_-wJN3YgGETANrGAdXJfd8NcUYJXBXLxsxplGdxT3AIxznjHGoWLvA-O6r_qI3ifs3jYtAEqNNX1oXU4LtOat8zBhMnpjbqUdkN9d_HScQ7bRqas Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 54761854b3e9496ea28cc4c8224c25bf Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesNegativeTest-34305390 Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:03:50.098529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/54761854b3e9496ea28cc4c8224c25bf"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/54761854b3e9496ea28cc4c8224c25bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_54761854b3e9496ea28cc4c8224c25bf"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/54761854b3e9496ea28cc4c8224c25bf"}]}] Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesNegativeTest-34305390 Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 54761854b3e9496ea28cc4c8224c25bf Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesNegativeTest-34305390 Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesNegativeTest-34305390-project Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 7d5aef87230d4424a73e87248ae81d78 Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesNegativeTest-34305390-project Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 8e28379c-2544-434d-bd34-794bccec8b71 could not be found. Sep 18 01:03:50.099927 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '8e28379c-2544-434d-bd34-794bccec8b71'} Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e28379c-2544-434d-bd34-794bccec8b71 could not be found. Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:50.100714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e28379c-2544-434d-bd34-794bccec8b71 could not be found. Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e28379c-2544-434d-bd34-794bccec8b71 could not be found. Sep 18 01:03:50.101990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:50.103110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] Returning 404 to user: Instance 8e28379c-2544-434d-bd34-794bccec8b71 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:50.103110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-bcb24fbd-f5ec-4dac-947f-2b6924c62b17 tempest-VolumesNegativeTest-34305390 tempest-VolumesNegativeTest-34305390-project] 10.222.0.42 "GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71" status: 404 len: 111 microversion: 2.1 time: 0.038287 Sep 18 01:03:50.103110 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 395/791] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:50 2022] GET /compute/v2.1/servers/8e28379c-2544-434d-bd34-794bccec8b71 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:50.219967 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:50.220731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:50.227627 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:50.228411 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:50.229038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:50.230575 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:50.231437 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:50.231905 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:50.396430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:50.405927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2f848e2a-e118-47c9-a300-428ea0673fa9 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.190327 Sep 18 01:03:50.406591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 397/792] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:50 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:50.928955 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:50.929563 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:50.936026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:50.936514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:50.936946 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:50.937639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:50.938091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:50.938534 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:50.970734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:50.971219 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:50.971651 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:50.972099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lazy-loading 'fault' on Instance uuid aeb33bda-3ec0-41d5-b1fd-edcabb801279 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:03:51.017386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f9751adf8d74ab59b6b0a5d00221405', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cf1375f9f28c43fda090a236635c4188', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:51.026023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3108c2ed-d516-465c-ac88-f0ca1b1eb79f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 200 len: 1330 microversion: 2.1 time: 0.100792 Sep 18 01:03:51.026619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 396/793] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:50 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 1330 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:51.434429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:51.436102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:51.445555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:51.446293 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:51.446994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:51.448214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:51.448845 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:51.449488 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:51.596642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:51.605387 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e2fef2bb-9461-415b-8714-668a07ffd87f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.179711 Sep 18 01:03:51.605953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 398/794] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:51 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:52.049285 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:52.049899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:52.056483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:52.056995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:52.057451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:52.058157 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:52.058640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:52.059063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:52.073821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 HTTP/1.1 Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:03:52.075371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:52.077009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'aeb33bda-3ec0-41d5-b1fd-edcabb801279'} Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:52.077931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:52.079535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:52.080843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:52.080843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-79cb10e5-4a19-4556-96f2-3f4af7e02ea7 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 404 len: 111 microversion: 2.1 time: 0.031276 Sep 18 01:03:52.080843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 397/795] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:52 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:52.086188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-70f6808a-45ee-48a5-a819-bc92c0e635e9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '.version_select at 0x7fcab1b12430>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:52.105700 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-70f6808a-45ee-48a5-a819-bc92c0e635e9 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-727572029" status: 202 len: 0 microversion: 2.1 time: 0.023539 Sep 18 01:03:52.106384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 399/796] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 18 01:03:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-727572029 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:03:52.627893 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:52.628615 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:52.635076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:52.636281 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:52.637079 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:52.637920 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:52.638496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:52.639153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:52.805387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:52.815354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-580e73b3-c44a-4051-87b3-7c3cc453cff2 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.191417 Sep 18 01:03:52.816071 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 398/797] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:52 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:53.841947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:53.842611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:53.849121 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:53.849600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:53.850037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:53.850742 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:53.851197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:53.851621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:53.933297 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:53.933943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:53.941042 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:53.941673 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:53.942136 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:53.943035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:53.943512 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:53.943953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:53.958609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: DELETE /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 HTTP/1.1 Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:03:53.960338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:53.962083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'aeb33bda-3ec0-41d5-b1fd-edcabb801279'} Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:53.963038 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:53.964972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:53.967209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:53.968046 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-44b8f603-3ff4-46bd-a3e9-e370c5e979e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 404 len: 111 microversion: 2.1 time: 0.039111 Sep 18 01:03:53.969205 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 399/798] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:03:53 2022] DELETE /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 01:03:53.978199 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:53.979776 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: aeb33bda-3ec0-41d5-b1fd-edcabb801279] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:53.986143 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:53.986957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:53.987875 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:53.988894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:53.989574 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:53.990101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:53.999925 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:54.005713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 HTTP/1.1 Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:03:54.007412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:54.009343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'aeb33bda-3ec0-41d5-b1fd-edcabb801279'} Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:54.010221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. Sep 18 01:03:54.011705 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:54.013235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-077959ed-b034-4994-9165-d1f0a28b78ee tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.169679 Sep 18 01:03:54.013235 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 400/799] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:53 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:54.013381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance aeb33bda-3ec0-41d5-b1fd-edcabb801279 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:54.013381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0311eefe-7d73-4a0e-a764-332d4fd1004f tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279" status: 404 len: 111 microversion: 2.1 time: 0.036344 Sep 18 01:03:54.013381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 400/800] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:53 2022] GET /compute/v2.1/servers/aeb33bda-3ec0-41d5-b1fd-edcabb801279 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:54.348637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-834bf01a-501e-4d75-bbc7-1012b5f5405f req-4be5e6d0-ba19-49dc-92dc-b67c207ca711 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4aaef7df-7526-47f3-a235-e7565d13df51", "name": "network-vif-plugged", "status": "completed", "tag": "d89c76f8-4e53-4830-b1fe-dff8b48e622b"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:54.362913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-834bf01a-501e-4d75-bbc7-1012b5f5405f req-4be5e6d0-ba19-49dc-92dc-b67c207ca711 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:54.363390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-834bf01a-501e-4d75-bbc7-1012b5f5405f req-4be5e6d0-ba19-49dc-92dc-b67c207ca711 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:54.363820 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-834bf01a-501e-4d75-bbc7-1012b5f5405f req-4be5e6d0-ba19-49dc-92dc-b67c207ca711 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:54.367833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:03:54.376815 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-834bf01a-501e-4d75-bbc7-1012b5f5405f req-4be5e6d0-ba19-49dc-92dc-b67c207ca711 service nova] Creating event network-vif-plugged:d89c76f8-4e53-4830-b1fe-dff8b48e622b for instance 4aaef7df-7526-47f3-a235-e7565d13df51 on hv-cinder-maste Sep 18 01:03:54.384478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-834bf01a-501e-4d75-bbc7-1012b5f5405f req-4be5e6d0-ba19-49dc-92dc-b67c207ca711 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040310 Sep 18 01:03:54.385801 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 401/801] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:03:55.030725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:55.031256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:55.037782 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.038264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.038738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.039430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.039887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.040315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.160988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:55.161576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:55.168883 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.169378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.169810 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.170552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.171134 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.171569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.185222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 HTTP/1.1 Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:03:55.186976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.188503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '6c8469ab-e2ff-405e-9da7-e4e93e64fc72'} Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:55.189330 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.190687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.191900 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:55.191900 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-450db643-ae0e-4b5d-aab6-d9f710922713 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 404 len: 111 microversion: 2.1 time: 0.032439 Sep 18 01:03:55.191900 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 402/802] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:03:55 2022] DELETE /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 01:03:55.196834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:55.197447 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 6c8469ab-e2ff-405e-9da7-e4e93e64fc72] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:55.199697 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:55.204102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.204594 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.205034 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.205737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.206378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.206833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.208019 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e0502357-0e5d-4a33-8c9c-396ecf7d4920 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1263 microversion: 2.1 time: 0.181754 Sep 18 01:03:55.208643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 401/803] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:55 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1263 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:55.221038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 HTTP/1.1 Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:03:55.222607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.224193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '6c8469ab-e2ff-405e-9da7-e4e93e64fc72'} Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:55.225135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. Sep 18 01:03:55.226504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.227805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 6c8469ab-e2ff-405e-9da7-e4e93e64fc72 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:55.227805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-68412f74-4fed-41ae-b7e3-92ca8afda1dc tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72" status: 404 len: 111 microversion: 2.1 time: 0.030860 Sep 18 01:03:55.227805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 403/804] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:55 2022] GET /compute/v2.1/servers/6c8469ab-e2ff-405e-9da7-e4e93e64fc72 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 01:03:55.232848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:55.233349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:55.239764 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.240280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.240720 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.241433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.241890 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.242317 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.255850 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: DELETE /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 HTTP/1.1 Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:03:55.257413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '1a2de33a-66be-4df0-9c70-adf5c1b03ec8'} Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:55.259907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.261710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.263185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:55.263185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-941f6670-8313-4a45-81ba-0070f1bb9a67 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 404 len: 111 microversion: 2.1 time: 0.029976 Sep 18 01:03:55.263185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 402/805] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:03:55 2022] DELETE /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:55.267946 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:55.268559 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 1a2de33a-66be-4df0-9c70-adf5c1b03ec8] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:55.274924 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.275456 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.275942 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.277241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:55.277699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:55.278156 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:55.291827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 HTTP/1.1 Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:03:55.293170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.294760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '1a2de33a-66be-4df0-9c70-adf5c1b03ec8'} Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:55.295651 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. Sep 18 01:03:55.297076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:03:55.298346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 1a2de33a-66be-4df0-9c70-adf5c1b03ec8 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:55.298346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-73a2cb1f-681f-414f-8bf4-9fe4dbfc54e1 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8" status: 404 len: 111 microversion: 2.1 time: 0.030490 Sep 18 01:03:55.298346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 404/806] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:55 2022] GET /compute/v2.1/servers/1a2de33a-66be-4df0-9c70-adf5c1b03ec8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:56.231623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:56.232243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:56.238957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.239430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.239864 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.240552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.241007 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.241475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.384241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:56.392149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e97c88d9-5d8d-4141-8169-1500f41b0041 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1451 microversion: 2.1 time: 0.164925 Sep 18 01:03:56.392750 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 403/807] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:56 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1451 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:56.414413 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:56.415055 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:56.422433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.422955 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.423394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.424094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.424550 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.424979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.433870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:56.434527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:56.440480 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.440967 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.441404 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.442093 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.442608 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.443039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.456834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: DELETE /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 HTTP/1.1 Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:03:56.458725 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.460098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '0368fcd1-89d7-4e1f-ad08-1425f3872064'} Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:56.460843 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.462192 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.463863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:56.463863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e2a9cfc8-c661-448b-a671-9c23e603c8e0 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "DELETE /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 404 len: 111 microversion: 2.1 time: 0.030252 Sep 18 01:03:56.463863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 404/808] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:03:56 2022] DELETE /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 18 01:03:56.469280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:03:56.469938 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] [instance: 0368fcd1-89d7-4e1f-ad08-1425f3872064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:03:56.475938 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.476454 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.476867 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.477571 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:56.478025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:56.478446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:56.493497 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] HTTP exception thrown: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Request method failure captured: Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 HTTP/1.1 Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzoNH5eaxD6TbGhiffQQAlFZX3S8IZ7BQR7rZM4WLctH92eivomaFgUF2377SOa31F0wY-2Qpftp415J9BxJtTtaF-maAijxW55GiiiYDSqOKFJvcxxPTp6DoKZ8hOU-Txu_ctMFIqSl-UtW8zV87dkNoHsWQE5TTJFfRoc3ERDa6E Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:03:56.495209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cf1375f9f28c43fda090a236635c4188"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cf1375f9f28c43fda090a236635c4188"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cf1375f9f28c43fda090a236635c4188"}]}] Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: cf1375f9f28c43fda090a236635c4188 Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1122085850 Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 2f9751adf8d74ab59b6b0a5d00221405 Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeNegativeTest-1122085850-project Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.496719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '0368fcd1-89d7-4e1f-ad08-1425f3872064'} Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:03:56.497490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. Sep 18 01:03:56.498758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:03:56.499899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] Returning 404 to user: Instance 0368fcd1-89d7-4e1f-ad08-1425f3872064 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:03:56.499899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d5656820-5674-4434-a6e2-4614abb29284 tempest-AttachVolumeNegativeTest-1122085850 tempest-AttachVolumeNegativeTest-1122085850-project] 10.222.0.42 "GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064" status: 404 len: 111 microversion: 2.1 time: 0.031090 Sep 18 01:03:56.499899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 405/809] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:56 2022] GET /compute/v2.1/servers/0368fcd1-89d7-4e1f-ad08-1425f3872064 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:03:56.808692 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:03:56.816515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f1e93de7-b0a9-4d56-8214-7a92e5e5b956 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1451 microversion: 2.1 time: 0.405820 Sep 18 01:03:56.817052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 405/810] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:03:56 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1451 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:03:58.457368 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-ead76c69-ed55-43b4-b08a-6f913797a210 req-eb38a8ab-10f5-44a8-b8c8-813531cb560d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4aaef7df-7526-47f3-a235-e7565d13df51", "tag": "d89c76f8-4e53-4830-b1fe-dff8b48e622b"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:03:58.472358 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ead76c69-ed55-43b4-b08a-6f913797a210 req-eb38a8ab-10f5-44a8-b8c8-813531cb560d service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:03:58.473100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ead76c69-ed55-43b4-b08a-6f913797a210 req-eb38a8ab-10f5-44a8-b8c8-813531cb560d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:03:58.473699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-ead76c69-ed55-43b4-b08a-6f913797a210 req-eb38a8ab-10f5-44a8-b8c8-813531cb560d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:03:58.486796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-ead76c69-ed55-43b4-b08a-6f913797a210 req-eb38a8ab-10f5-44a8-b8c8-813531cb560d service nova] Creating event network-changed:d89c76f8-4e53-4830-b1fe-dff8b48e622b for instance 4aaef7df-7526-47f3-a235-e7565d13df51 on hv-cinder-maste Sep 18 01:03:58.492997 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-ead76c69-ed55-43b4-b08a-6f913797a210 req-eb38a8ab-10f5-44a8-b8c8-813531cb560d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039137 Sep 18 01:03:58.498872 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 406/811] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:03:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:01.268225 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-54984956-9718-4593-8bba-f7a942273ddc tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Action: 'create', calling method: .version_select at 0x7fcab1ce6dc0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-859490438"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:01.573491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-54984956-9718-4593-8bba-f7a942273ddc tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.422081 Sep 18 01:04:01.574177 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 406/812] 10.222.0.42 () {60 vars in 1155 bytes} [Sun Sep 18 01:04:01 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 423 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 18 01:04:01.589969 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0f777fa2-5315-4d60-9e6d-6db5b5329774 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab19dd550>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:01.638738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0f777fa2-5315-4d60-9e6d-6db5b5329774 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f" status: 200 len: 344 microversion: 2.1 time: 0.053270 Sep 18 01:04:01.639310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 407/813] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:04:01 2022] GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f => generated 344 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:02.653227 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f521aa4c-0f87-4a44-8e57-564637295fb1 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1b064c0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:02.700945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f521aa4c-0f87-4a44-8e57-564637295fb1 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f" status: 200 len: 345 microversion: 2.1 time: 0.052091 Sep 18 01:04:02.701501 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 407/814] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:04:02 2022] GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f => generated 345 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:02.715122 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6fda6e3a-1edc-4c00-881b-419caf46afb1 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Action: 'create', calling method: .version_select at 0x7fcab1a24280>, body: {"snapshot": {"volume_id": "7f332e92-0afd-497d-a02c-656b6b5c802f", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1289103136"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:02.863941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6fda6e3a-1edc-4c00-881b-419caf46afb1 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.152905 Sep 18 01:04:02.864778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 408/815] 10.222.0.42 () {60 vars in 1162 bytes} [Sun Sep 18 01:04:02 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:02.877176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-afd5cab1-ae18-4422-8664-e303193acdcf tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1b06ee0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:02.909740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-afd5cab1-ae18-4422-8664-e303193acdcf tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 200 len: 287 microversion: 2.1 time: 0.037084 Sep 18 01:04:02.910292 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 408/816] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 18 01:04:02 2022] GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:02.921663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d2dfcd5e-1be1-474e-8143-7024bc4a4d4a tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1a25a60>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:02.952319 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d2dfcd5e-1be1-474e-8143-7024bc4a4d4a tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.035063 Sep 18 01:04:02.952914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 409/817] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Sep 18 01:04:02 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:02.964090 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-547b3790-adff-4d63-8913-27c48cae6b0d tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1b06b80>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:02.993687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-547b3790-adff-4d63-8913-27c48cae6b0d tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 200 len: 287 microversion: 2.1 time: 0.033325 Sep 18 01:04:02.994241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 409/818] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 18 01:04:02 2022] GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 287 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:04.007028 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a494144a-a8db-466b-b4df-85a8ca1be7fc tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1ac7dc0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:04.036578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a494144a-a8db-466b-b4df-85a8ca1be7fc tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 200 len: 287 microversion: 2.1 time: 0.033836 Sep 18 01:04:04.037150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 410/819] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 18 01:04:04 2022] GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:05.049411 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-502fb954-d0a4-4e1a-a9cd-44b0a33367f9 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1aed9d0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:05.081934 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-502fb954-d0a4-4e1a-a9cd-44b0a33367f9 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 200 len: 287 microversion: 2.1 time: 0.036837 Sep 18 01:04:05.082521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 410/820] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 18 01:04:05 2022] GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:06.095274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d72b53ef-4c9b-4be6-89a5-9fe2866a9cf2 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1b7f550>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:06.125490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d72b53ef-4c9b-4be6-89a5-9fe2866a9cf2 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 200 len: 288 microversion: 2.1 time: 0.034705 Sep 18 01:04:06.126088 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 411/821] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 18 01:04:06 2022] GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 288 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:06.138284 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a356d2f4-47f4-4f67-a73b-162d6b497ca3 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1b4adc0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:06.195304 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a356d2f4-47f4-4f67-a73b-162d6b497ca3 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "DELETE /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 202 len: 0 microversion: 2.1 time: 0.061849 Sep 18 01:04:06.195899 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 411/822] 10.222.0.42 () {58 vars in 1254 bytes} [Sun Sep 18 01:04:06 2022] DELETE /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 0 bytes in 63 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:04:06.204831 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d56e8d0e-baa9-4623-a6a9-2c651c3b4f54 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab19dd280>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:06.235195 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d56e8d0e-baa9-4623-a6a9-2c651c3b4f54 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 200 len: 287 microversion: 2.1 time: 0.034220 Sep 18 01:04:06.235756 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 412/823] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 18 01:04:06 2022] GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:04:07.246273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=30,nova_api:DELETE=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:07.247276 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=13 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:07.251785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5d76e345-714a-45ab-9c42-52e22c8a1271 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1aed1f0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:07.276892 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-5d76e345-714a-45ab-9c42-52e22c8a1271 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] HTTP exception thrown: Snapshot 9686d407-4000-40cd-a517-572a80c32027 could not be found. Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5d76e345-714a-45ab-9c42-52e22c8a1271 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Request method failure captured: Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 HTTP/1.1 Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm6BhdIgpF8vgQd49C4CDeclrcu7-knb4uEJ7TtrE8m7dGRqTDZ2AS6c-nTpycMLv4LbLRC6D92lFsPSfWJ5IUkkRMbebFFlIQbMq5Rt82f421k5L-MGy11bxKAVezmWiW7EP6wWWk8Bpk-3Es43Xj3D6TGGnN324bDe33qh4wEmpU4 Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:04:07.279176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_7806d126c00c41dabdc84f853f13becc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}] Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: d19b2f92e46148c5a3d53c3ddffede77 Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Snapshot 9686d407-4000-40cd-a517-572a80c32027 could not be found. Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '9686d407-4000-40cd-a517-572a80c32027'} Sep 18 01:04:07.280794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 9686d407-4000-40cd-a517-572a80c32027 could not be found. (HTTP 404) (Request-ID: req-fa8453cf-5e3a-406d-8649-2a2c6b01e152) Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:07.282058 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:04:07.283520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 9686d407-4000-40cd-a517-572a80c32027 could not be found. Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 18 01:04:07.285018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 9686d407-4000-40cd-a517-572a80c32027 could not be found. Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 9686d407-4000-40cd-a517-572a80c32027 could not be found. Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5d76e345-714a-45ab-9c42-52e22c8a1271 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Returning 404 to user: Snapshot 9686d407-4000-40cd-a517-572a80c32027 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:07.286571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5d76e345-714a-45ab-9c42-52e22c8a1271 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027" status: 404 len: 111 microversion: 2.1 time: 0.035871 Sep 18 01:04:07.287546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 412/824] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 18 01:04:07 2022] GET /compute/v2.1/os-snapshots/9686d407-4000-40cd-a517-572a80c32027 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:04:07.290075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-88654c53-a88c-4b6a-8ae9-e8eae0bc8b36 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1ac74c0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:07.354620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-88654c53-a88c-4b6a-8ae9-e8eae0bc8b36 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f" status: 202 len: 0 microversion: 2.1 time: 0.068776 Sep 18 01:04:07.355579 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 413/825] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:04:07 2022] DELETE /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:04:07.364622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cca3d751-309b-4e92-a0f2-9ef3bcfb2beb tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1b06700>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:07.412986 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cca3d751-309b-4e92-a0f2-9ef3bcfb2beb tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f" status: 200 len: 344 microversion: 2.1 time: 0.052944 Sep 18 01:04:07.413566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 413/826] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:04:07 2022] GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f => generated 344 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:08.429090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c84637b8-9970-437f-839b-23770867c77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab19dd3a0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:08.459462 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-c84637b8-9970-437f-839b-23770867c77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] HTTP exception thrown: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c84637b8-9970-437f-839b-23770867c77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Request method failure captured: Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f HTTP/1.1 Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm6BhdIgpF8vgQd49C4CDeclrcu7-knb4uEJ7TtrE8m7dGRqTDZ2AS6c-nTpycMLv4LbLRC6D92lFsPSfWJ5IUkkRMbebFFlIQbMq5Rt82f421k5L-MGy11bxKAVezmWiW7EP6wWWk8Bpk-3Es43Xj3D6TGGnN324bDe33qh4wEmpU4 Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:04:08.461548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_7806d126c00c41dabdc84f853f13becc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}] Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: d19b2f92e46148c5a3d53c3ddffede77 Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '7f332e92-0afd-497d-a02c-656b6b5c802f'} Sep 18 01:04:08.463101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. (HTTP 404) (Request-ID: req-ff52b17f-b907-4449-88aa-fc5cecc4536e) Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.464047 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:04:08.465451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 18 01:04:08.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c84637b8-9970-437f-839b-23770867c77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Returning 404 to user: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c84637b8-9970-437f-839b-23770867c77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f" status: 404 len: 109 microversion: 2.1 time: 0.038359 Sep 18 01:04:08.468207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 414/827] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:04:08 2022] GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:04:08.477511 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e1d3cfc3-56da-4b1e-bad7-bf4d6b7bf77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab1b06b80>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:08.504976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-e1d3cfc3-56da-4b1e-bad7-bf4d6b7bf77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] HTTP exception thrown: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e1d3cfc3-56da-4b1e-bad7-bf4d6b7bf77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Request method failure captured: Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f HTTP/1.1 Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm6BhdIgpF8vgQd49C4CDeclrcu7-knb4uEJ7TtrE8m7dGRqTDZ2AS6c-nTpycMLv4LbLRC6D92lFsPSfWJ5IUkkRMbebFFlIQbMq5Rt82f421k5L-MGy11bxKAVezmWiW7EP6wWWk8Bpk-3Es43Xj3D6TGGnN324bDe33qh4wEmpU4 Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:04:08.506969 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_7806d126c00c41dabdc84f853f13becc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}] Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: d19b2f92e46148c5a3d53c3ddffede77 Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '7f332e92-0afd-497d-a02c-656b6b5c802f'} Sep 18 01:04:08.508475 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. (HTTP 404) (Request-ID: req-dfc3baf3-fc3d-442d-9351-b88ac528d62e) Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 18 01:04:08.514237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 18 01:04:08.515542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e1d3cfc3-56da-4b1e-bad7-bf4d6b7bf77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Returning 404 to user: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e1d3cfc3-56da-4b1e-bad7-bf4d6b7bf77f tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f" status: 404 len: 109 microversion: 2.1 time: 0.035094 Sep 18 01:04:08.516658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 414/828] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:04:08 2022] DELETE /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:04:08.517535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:08.517535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=50 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:08.520931 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8f8e03e0-2615-44b0-80c5-c8732896b709 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Calling method '.version_select at 0x7fcab19dd5e0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:08.550581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-8f8e03e0-2615-44b0-80c5-c8732896b709 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] HTTP exception thrown: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8f8e03e0-2615-44b0-80c5-c8732896b709 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Request method failure captured: Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f HTTP/1.1 Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm6BhdIgpF8vgQd49C4CDeclrcu7-knb4uEJ7TtrE8m7dGRqTDZ2AS6c-nTpycMLv4LbLRC6D92lFsPSfWJ5IUkkRMbebFFlIQbMq5Rt82f421k5L-MGy11bxKAVezmWiW7EP6wWWk8Bpk-3Es43Xj3D6TGGnN324bDe33qh4wEmpU4 Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:04:08.551964 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/7806d126c00c41dabdc84f853f13becc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_7806d126c00c41dabdc84f853f13becc"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/7806d126c00c41dabdc84f853f13becc"}]}] Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 7806d126c00c41dabdc84f853f13becc Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1250306497 Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: d19b2f92e46148c5a3d53c3ddffede77 Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1250306497-project Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '7f332e92-0afd-497d-a02c-656b6b5c802f'} Sep 18 01:04:08.553581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. (HTTP 404) (Request-ID: req-978a15af-a78d-482a-945b-c855c9032547) Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.554834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 18 01:04:08.556290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 18 01:04:08.557771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8f8e03e0-2615-44b0-80c5-c8732896b709 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] Returning 404 to user: Volume 7f332e92-0afd-497d-a02c-656b6b5c802f could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8f8e03e0-2615-44b0-80c5-c8732896b709 tempest-VolumesSnapshotsTestJSON-1250306497 tempest-VolumesSnapshotsTestJSON-1250306497-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f" status: 404 len: 109 microversion: 2.1 time: 0.039572 Sep 18 01:04:08.559741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 415/829] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:04:08 2022] GET /compute/v2.1/os-volumes/7f332e92-0afd-497d-a02c-656b6b5c802f => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:04:15.272248 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:15.274501 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:15.280532 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:15.281114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:15.281457 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:15.282155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:15.282649 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:15.283078 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:15.312931 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c90c47b9-d1b3-4cdb-8485-3cfb9de5b95c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.047369 Sep 18 01:04:15.313515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 415/830] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 18 01:04:15 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:15.378896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:15.379498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:15.386752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:15.387244 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:15.387712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:15.388414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:15.388866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:15.389290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:15.414382 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:04:15.496155 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-63f3366a-4cb3-434e-9720-351c38254e66 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 204 len: 0 microversion: 2.20 time: 0.120473 Sep 18 01:04:15.497023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 416/831] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 18 01:04:15 2022] DELETE /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 01:04:15.505673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:15.506489 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:15.512658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:15.513264 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:15.513708 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:15.514735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:15.515098 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:15.515525 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:15.649642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:15.658173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:15.659699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:15.660858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7ac8a118-3dd9-4f0e-92bc-e56de011ee73 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.159231 Sep 18 01:04:15.661419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 416/832] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:15 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:04:16.687376 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:16.687991 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:16.694334 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:16.694857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:16.695292 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:16.695978 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:16.696436 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:16.696860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:16.829512 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:16.837811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:16.839894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:16.841069 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-66257a06-5981-40f0-9eec-4f1698b97669 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1724 microversion: 2.20 time: 0.158039 Sep 18 01:04:16.841793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 417/833] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:16 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:04:17.867152 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:17.867890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:17.876010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:17.876507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:17.876988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:17.877751 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:17.878276 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:17.878787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:18.148898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:18.158299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:18.159796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:18.160953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9998f125-4340-46be-961f-da9bdc1cef9a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1657 microversion: 2.20 time: 0.297815 Sep 18 01:04:18.161506 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 417/834] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:17 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1657 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:04:18.379312 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-e5c76554-e8d0-4974-b275-50873a33bc67 req-b33d4055-2af9-430c-baea-61a2130c1e6d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a40bb3d-0090-4ce2-9782-c61f0161e064", "name": "network-vif-deleted", "tag": "5025d940-e81a-4c19-8fda-e4fac2888b2a"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:18.391566 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e5c76554-e8d0-4974-b275-50873a33bc67 req-b33d4055-2af9-430c-baea-61a2130c1e6d service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:18.392042 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e5c76554-e8d0-4974-b275-50873a33bc67 req-b33d4055-2af9-430c-baea-61a2130c1e6d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:18.392465 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e5c76554-e8d0-4974-b275-50873a33bc67 req-b33d4055-2af9-430c-baea-61a2130c1e6d service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:18.405237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-e5c76554-e8d0-4974-b275-50873a33bc67 req-b33d4055-2af9-430c-baea-61a2130c1e6d service nova] Creating event network-vif-deleted:5025d940-e81a-4c19-8fda-e4fac2888b2a for instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 on hv-cinder-maste Sep 18 01:04:18.410239 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-e5c76554-e8d0-4974-b275-50873a33bc67 req-b33d4055-2af9-430c-baea-61a2130c1e6d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034485 Sep 18 01:04:18.410817 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 418/835] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:04:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:04:19.187787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:19.188494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:19.195495 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:19.195974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:19.196405 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:19.197102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:19.197553 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:19.198013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:19.262994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:19.271565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:19.273052 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:19.274207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1126ef53-6787-479f-8da5-27c949e717ab tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1657 microversion: 2.20 time: 0.090949 Sep 18 01:04:19.274791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 418/836] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:19 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1657 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:04:20.300595 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:20.301288 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:20.307657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:20.308651 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:20.309088 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:20.309780 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:20.310235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:20.310691 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:20.372320 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:20.380471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:20.382040 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '294ae98a8492468893beb3b8c2870d5e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '505bd70415a04f53a64645eb7959a5fb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:20.383338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a997bca8-7d89-4614-9802-0dd2093be076 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 200 len: 1366 microversion: 2.20 time: 0.087489 Sep 18 01:04:20.383915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 419/837] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:20 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 1366 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 18 01:04:21.409269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:21.409878 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:21.416301 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:21.416823 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:21.417261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:21.417957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:21.418415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:21.418879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:21.432789 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] HTTP exception thrown: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Request method failure captured: Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 HTTP/1.1 Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzmaso6D5zTG4pZx5n1a7YeINaSoMRjyPDFIzt6YRAF1nyLLD5VG506tTehjPTjn5tXUtJYrbv0QwuLl0EK5wvD84ApKctiQlJ_EQaLnOIgCLEQVD82q7MrE8lKvBj3E-XoYpBdI5Uua4R9pUyQ3IfLhhX05m1BgPYuSan8QhxRxnw Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:04:21.434577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}] Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 294ae98a8492468893beb3b8c2870d5e Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:21.436445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '8a40bb3d-0090-4ce2-9782-c61f0161e064'} Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:21.437452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:21.439015 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:21.440535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Returning 404 to user: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:21.440535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6a566b42-b25d-4c05-aa96-b52e18e09bb0 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 404 len: 111 microversion: 2.20 time: 0.030606 Sep 18 01:04:21.440535 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 419/838] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:21 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:04:21.445740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cec86df8-319e-4904-b21e-a60e902f832c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '.version_select at 0x7fcab1a48940>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:21.466903 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cec86df8-319e-4904-b21e-a60e902f832c tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1218396652" status: 204 len: 0 microversion: 2.20 time: 0.024165 Sep 18 01:04:21.467482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 420/839] 10.222.0.42 () {60 vars in 1262 bytes} [Sun Sep 18 01:04:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1218396652 => generated 0 bytes in 26 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 18 01:04:23.224165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:23.224647 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:23.231754 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:23.232229 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:23.232662 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:23.233352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:23.233798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:23.234219 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:23.247625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] HTTP exception thrown: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Request method failure captured: Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 HTTP/1.1 Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzmaso6D5zTG4pZx5n1a7YeINaSoMRjyPDFIzt6YRAF1nyLLD5VG506tTehjPTjn5tXUtJYrbv0QwuLl0EK5wvD84ApKctiQlJ_EQaLnOIgCLEQVD82q7MrE8lKvBj3E-XoYpBdI5Uua4R9pUyQ3IfLhhX05m1BgPYuSan8QhxRxnw Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:04:23.249157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}] Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 294ae98a8492468893beb3b8c2870d5e Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.250839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '8a40bb3d-0090-4ce2-9782-c61f0161e064'} Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:23.251735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.253273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.254797 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Returning 404 to user: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:23.254797 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c86ced30-1b8a-4349-b3c3-5b970a866954 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 404 len: 111 microversion: 2.20 time: 0.030262 Sep 18 01:04:23.254797 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 420/840] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 18 01:04:23 2022] DELETE /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:04:23.259594 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:23.260203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 8a40bb3d-0090-4ce2-9782-c61f0161e064] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:23.266856 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:23.267346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:23.267778 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:23.268469 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:23.269224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:23.269685 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:23.283532 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] HTTP exception thrown: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Request method failure captured: Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 HTTP/1.1 Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzmaso6D5zTG4pZx5n1a7YeINaSoMRjyPDFIzt6YRAF1nyLLD5VG506tTehjPTjn5tXUtJYrbv0QwuLl0EK5wvD84ApKctiQlJ_EQaLnOIgCLEQVD82q7MrE8lKvBj3E-XoYpBdI5Uua4R9pUyQ3IfLhhX05m1BgPYuSan8QhxRxnw Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:04:23.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}] Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 294ae98a8492468893beb3b8c2870d5e Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.286895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '8a40bb3d-0090-4ce2-9782-c61f0161e064'} Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:23.287761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. Sep 18 01:04:23.289303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:23.293189 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Returning 404 to user: Instance 8a40bb3d-0090-4ce2-9782-c61f0161e064 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:23.293189 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b53a99c2-8032-4f76-bf75-d59fe497ce84 tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064" status: 404 len: 111 microversion: 2.20 time: 0.031349 Sep 18 01:04:23.293189 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 421/841] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:23 2022] GET /compute/v2.1/servers/8a40bb3d-0090-4ce2-9782-c61f0161e064 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 18 01:04:24.428023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:24.428533 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:24.434984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:24.435469 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:24.435903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:24.436599 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:24.437050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:24.437469 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:24.451491 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] HTTP exception thrown: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Request method failure captured: Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 HTTP/1.1 Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJmzmaso6D5zTG4pZx5n1a7YeINaSoMRjyPDFIzt6YRAF1nyLLD5VG506tTehjPTjn5tXUtJYrbv0QwuLl0EK5wvD84ApKctiQlJ_EQaLnOIgCLEQVD82q7MrE8lKvBj3E-XoYpBdI5Uua4R9pUyQ3IfLhhX05m1BgPYuSan8QhxRxnw Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:04:24.453072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}] Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 294ae98a8492468893beb3b8c2870d5e Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.454564 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '5efa25bd-04ff-49f6-9fd5-6e8c82bcd152'} Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:24.455384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.456601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.457889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Returning 404 to user: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:24.457889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-90496f2d-6103-41cd-8a72-49159694237a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "DELETE /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 404 len: 111 microversion: 2.20 time: 0.030313 Sep 18 01:04:24.457889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 421/842] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 18 01:04:24 2022] DELETE /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:04:24.463895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:24.464502 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] [instance: 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:24.471471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:24.471937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:24.472367 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:24.473086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:24.473530 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:24.473948 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:24.487403 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] HTTP exception thrown: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Request method failure captured: Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 HTTP/1.1 Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJmzmaso6D5zTG4pZx5n1a7YeINaSoMRjyPDFIzt6YRAF1nyLLD5VG506tTehjPTjn5tXUtJYrbv0QwuLl0EK5wvD84ApKctiQlJ_EQaLnOIgCLEQVD82q7MrE8lKvBj3E-XoYpBdI5Uua4R9pUyQ3IfLhhX05m1BgPYuSan8QhxRxnw Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Openstack-Nova-Api-Version: 2.20 Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:04:24.488932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_505bd70415a04f53a64645eb7959a5fb"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/505bd70415a04f53a64645eb7959a5fb"}]}] Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 505bd70415a04f53a64645eb7959a5fb Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1811246332 Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 294ae98a8492468893beb3b8c2870d5e Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1811246332-project Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.490542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '5efa25bd-04ff-49f6-9fd5-6e8c82bcd152'} Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:04:24.491299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. Sep 18 01:04:24.492622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:04:24.493811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] Returning 404 to user: Instance 5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:04:24.493811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-977e8a30-f197-48d8-be3d-d9359971150a tempest-AttachVolumeShelveTestJSON-1811246332 tempest-AttachVolumeShelveTestJSON-1811246332-project] 10.222.0.42 "GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152" status: 404 len: 111 microversion: 2.20 time: 0.030359 Sep 18 01:04:24.493811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 422/843] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 18 01:04:24 2022] GET /compute/v2.1/servers/5efa25bd-04ff-49f6-9fd5-6e8c82bcd152 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 18 01:04:33.894927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c0698ba4-6138-47a0-ac2d-799796e78ce1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'create', calling method: .version_select at 0x7fcab1dcf670>, body: {"keypair": {"name": "tempest-keypair-1281574759", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA4WNHbYa7av6smpawCNmPWNIfmyyEyc7T0yG/xu5IxolFTSo/FzkxGtbae5sb+vlYAwDCS7GkZDj4vk+etaZYlEles7EgGKP5MOAMP+iNm3bnXw2U+kKSl3/hdZevB7CQ=="}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:33.916466 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c0698ba4-6138-47a0-ac2d-799796e78ce1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:04:33.921149 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c0698ba4-6138-47a0-ac2d-799796e78ce1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for user bd2fa2deb0a24aff9d8b7ce97b6e461b and project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:04:33.942934 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c0698ba4-6138-47a0-ac2d-799796e78ce1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.322644 Sep 18 01:04:33.950210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 422/844] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 01:04:33 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:35.585015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1214803033", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5e1cdc84-fae8-4064-bacf-43638170a453"}], "security_groups": [{"name": "tempest-securitygroup--122165615"}], "key_name": "tempest-keypair-1281574759", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:35.594714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:35.595368 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=32,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:35.690449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] validate_networks() for [('5e1cdc84-fae8-4064-bacf-43638170a453', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:04:36.066865 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:04:36.067414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:04:36.067955 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:04:36.068438 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:04:36.068928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:04:36.079048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:36.079528 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:36.080098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:36.080686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:36.081144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:36.081567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:36.098863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:04:36.103132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for user bd2fa2deb0a24aff9d8b7ce97b6e461b and project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:04:36.114908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:04:36.117521 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:04:36.163141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-83b2d2a1-b473-460e-ada4-cad1e94f11ff tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.582697 Sep 18 01:04:36.163781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 423/845] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:04:35 2022] POST /compute/v2.1/servers => generated 401 bytes in 584 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:04:36.174851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-44aa18b7-7e09-484f-a1cd-bc3447f28944 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:36.175472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-44aa18b7-7e09-484f-a1cd-bc3447f28944 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:36.177441 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=30 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:36.232043 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-44aa18b7-7e09-484f-a1cd-bc3447f28944 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:36.238935 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-44aa18b7-7e09-484f-a1cd-bc3447f28944 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1238 microversion: 2.1 time: 0.067854 Sep 18 01:04:36.239490 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 423/846] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:36 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1238 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:37.262429 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:37.263061 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:37.270248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:37.270767 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:37.271205 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:37.271913 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:37.272457 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:37.272790 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:37.336310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:37.344639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0eece368-3818-4ebe-9d51-be37fd4ccae9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1290 microversion: 2.1 time: 0.086765 Sep 18 01:04:37.345210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 424/847] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:37 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1290 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:38.369427 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:38.372371 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:38.379474 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:38.381383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:38.382287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:38.382940 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:38.384113 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:38.384767 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:38.385385 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:38.475418 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:38.484421 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5b1a8f27-4486-46f6-84c8-958cae128265 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1296 microversion: 2.1 time: 0.119719 Sep 18 01:04:38.496954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 424/848] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:38 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1296 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:39.453650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-73e6ac60-6593-483a-8c65-086d5ff83088 req-4daef7f0-2906-42fd-bbc9-26498866e510 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "529a4d99-a333-4068-a419-09d67a1ae170", "tag": "8a184e82-50ee-43b7-b12b-caa0631860c8"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:39.466216 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-73e6ac60-6593-483a-8c65-086d5ff83088 req-4daef7f0-2906-42fd-bbc9-26498866e510 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:39.466818 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-73e6ac60-6593-483a-8c65-086d5ff83088 req-4daef7f0-2906-42fd-bbc9-26498866e510 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:39.467783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-73e6ac60-6593-483a-8c65-086d5ff83088 req-4daef7f0-2906-42fd-bbc9-26498866e510 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:39.481907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-73e6ac60-6593-483a-8c65-086d5ff83088 req-4daef7f0-2906-42fd-bbc9-26498866e510 service nova] Creating event network-changed:8a184e82-50ee-43b7-b12b-caa0631860c8 for instance 529a4d99-a333-4068-a419-09d67a1ae170 on hv-cinder-maste Sep 18 01:04:39.489515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-73e6ac60-6593-483a-8c65-086d5ff83088 req-4daef7f0-2906-42fd-bbc9-26498866e510 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039058 Sep 18 01:04:39.490501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 425/849] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:04:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:39.508194 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:39.508811 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:39.516071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:39.516566 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:39.516994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:39.517982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:39.519083 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:39.519686 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:39.740416 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:39.749257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-10f2c5e6-3fef-41db-b311-ce8bdeefa492 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1363 microversion: 2.1 time: 0.245117 Sep 18 01:04:39.749826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 425/850] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:39 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1363 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:40.772392 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:40.773092 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:40.779740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:40.780222 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:40.780667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:40.781367 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:40.781819 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:40.782242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:41.019162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:41.026830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-531d7903-9dfc-4e57-933c-e09bdd1ea205 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1363 microversion: 2.1 time: 0.258739 Sep 18 01:04:41.027493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 426/851] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:40 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1363 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:42.051683 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:42.053712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:42.060862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:42.061340 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:42.061774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:42.062486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:42.062933 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:42.063353 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:42.201011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:42.208620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-35f41ae1-d33e-47c3-9163-56b81802902e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1363 microversion: 2.1 time: 0.160797 Sep 18 01:04:42.209178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 426/852] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:42 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1363 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:43.232046 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:43.233202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:43.240248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:43.240668 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:43.241102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:43.241994 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:43.242448 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:43.243062 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:43.391037 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:43.398882 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a5e3f0ba-12e7-4f32-8a9d-cbf603942be3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1363 microversion: 2.1 time: 0.171094 Sep 18 01:04:43.399448 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 427/853] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:43 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1363 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:44.423371 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:44.423950 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:44.430404 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:44.430906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:44.431331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:44.432017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:44.432461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:44.432875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:44.573086 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:44.581275 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-30d9a182-4097-4dc6-9b9f-0bc0a68e9cba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1363 microversion: 2.1 time: 0.161754 Sep 18 01:04:44.581998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 427/854] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:44 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:44.868265 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-8da8106c-d7f8-475f-93c8-784c26d902df req-87868d6a-9c75-4639-8c38-8fd29d8b1f5e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "529a4d99-a333-4068-a419-09d67a1ae170", "name": "network-vif-plugged", "status": "completed", "tag": "8a184e82-50ee-43b7-b12b-caa0631860c8"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:44.880255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-8da8106c-d7f8-475f-93c8-784c26d902df req-87868d6a-9c75-4639-8c38-8fd29d8b1f5e service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:44.880738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-8da8106c-d7f8-475f-93c8-784c26d902df req-87868d6a-9c75-4639-8c38-8fd29d8b1f5e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:44.881171 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-8da8106c-d7f8-475f-93c8-784c26d902df req-87868d6a-9c75-4639-8c38-8fd29d8b1f5e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:44.892367 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-8da8106c-d7f8-475f-93c8-784c26d902df req-87868d6a-9c75-4639-8c38-8fd29d8b1f5e service nova] Creating event network-vif-plugged:8a184e82-50ee-43b7-b12b-caa0631860c8 for instance 529a4d99-a333-4068-a419-09d67a1ae170 on hv-cinder-maste Sep 18 01:04:44.897370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-8da8106c-d7f8-475f-93c8-784c26d902df req-87868d6a-9c75-4639-8c38-8fd29d8b1f5e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032948 Sep 18 01:04:44.898172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 428/855] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:04:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:45.604619 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:45.605405 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:45.612939 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:45.613432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:45.613873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:45.614618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:45.615076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:45.615532 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:45.766295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:45.774608 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a03cd77b-cc8f-4310-b99d-679f59fb90c3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1363 microversion: 2.1 time: 0.173816 Sep 18 01:04:45.775207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 428/856] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:45 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1363 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:46.799838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:46.800853 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:46.803379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:46.804054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=6 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:46.811999 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:46.813137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:46.813586 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:46.814591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:46.815055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:46.815477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:46.948434 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:04:46.956279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-136453bb-6958-44f3-8a3d-25364af1253f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1548 microversion: 2.1 time: 0.160763 Sep 18 01:04:46.956944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 429/857] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:04:46 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1548 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:04:46.978690 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:04:46.979229 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:04:46.981592 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:04:46.987394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:46.987879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:46.988314 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:46.989011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:46.989461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:46.989886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:47.067657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-91eb029f-b6a1-4067-be14-14356c67e9df tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092829 Sep 18 01:04:47.068308 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 429/858] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 01:04:46 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-interface => generated 284 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:04:48.195308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-afa6804d-bf38-48cc-8522-0c22a151c0fd req-950a2694-50ac-456e-ad94-d9f8d656ce9f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "529a4d99-a333-4068-a419-09d67a1ae170", "tag": "8a184e82-50ee-43b7-b12b-caa0631860c8"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:04:48.209012 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-afa6804d-bf38-48cc-8522-0c22a151c0fd req-950a2694-50ac-456e-ad94-d9f8d656ce9f service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:04:48.210232 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-afa6804d-bf38-48cc-8522-0c22a151c0fd req-950a2694-50ac-456e-ad94-d9f8d656ce9f service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:04:48.211090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-afa6804d-bf38-48cc-8522-0c22a151c0fd req-950a2694-50ac-456e-ad94-d9f8d656ce9f service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:04:48.224359 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-afa6804d-bf38-48cc-8522-0c22a151c0fd req-950a2694-50ac-456e-ad94-d9f8d656ce9f service nova] Creating event network-changed:8a184e82-50ee-43b7-b12b-caa0631860c8 for instance 529a4d99-a333-4068-a419-09d67a1ae170 on hv-cinder-maste Sep 18 01:04:48.231519 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-afa6804d-bf38-48cc-8522-0c22a151c0fd req-950a2694-50ac-456e-ad94-d9f8d656ce9f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039000 Sep 18 01:04:48.232609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 430/859] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:04:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:05:19.980782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=11 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:19.981593 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=34 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:20.183671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1cd7b9b1-fdf7-48bf-9063-9c539064e7bf tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Action: 'create', calling method: .version_select at 0x7fcab1b34d30>, body: {"keypair": {"name": "tempest-TestShelveInstance-1667373744", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG1KQfd6Uxdyau9nHPpwIMMw/eF4c2s+jWjSmywrrB6R1Ud3BKlrxzp7LpnTmS2Z2u6RtN1oLei9/CY3YZ6s04QrHmgPNbF31oUoDqWsq7yncckLqzOkV7WpntSsb5FJyw=="}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:20.200232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-1cd7b9b1-fdf7-48bf-9063-9c539064e7bf tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Getting quotas for project 2d812ad5fce24a49b0f2d07081d5718a. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:05:20.204850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-1cd7b9b1-fdf7-48bf-9063-9c539064e7bf tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Getting quotas for user 7ea84c1e0dcf441fb1346eaaef4de421 and project 2d812ad5fce24a49b0f2d07081d5718a. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:05:20.227865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1cd7b9b1-fdf7-48bf-9063-9c539064e7bf tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.248387 Sep 18 01:05:20.228471 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 430/860] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 01:05:19 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:05:24.188796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:24.189539 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=32 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:24.194099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:24.194715 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:24.202414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:24.202937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:24.203590 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:24.204305 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:24.204766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:24.205196 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:24.230723 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8fe16e05-2852-43d0-8a4b-bab8c113911f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/ips" status: 200 len: 144 microversion: 2.1 time: 0.043519 Sep 18 01:05:24.231520 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 431/861] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:05:24 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/ips => generated 144 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:05:26.685666 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5cc7c857-43b1-423f-ab8e-6cb3d2f2a9d2"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:26.688070 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:26.694893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:26.695377 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:26.695803 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:26.696487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:26.696938 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:26.697553 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:26.977130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-22a1fdd2-835e-4092-acb1-144cedc72c06 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.296564 Sep 18 01:05:26.977903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 431/862] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:05:26 2022] POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-volume_attachments => generated 194 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:05:34.524444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-44799004-4f6a-4392-b7a5-8bb77ac8d684 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:34.526637 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:34.527287 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=4 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:34.533669 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44799004-4f6a-4392-b7a5-8bb77ac8d684 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:34.534154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44799004-4f6a-4392-b7a5-8bb77ac8d684 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:34.534623 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-44799004-4f6a-4392-b7a5-8bb77ac8d684 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:34.600421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-44799004-4f6a-4392-b7a5-8bb77ac8d684 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Going to try to stop instance {{(pid=120831) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 18 01:05:34.643413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-44799004-4f6a-4392-b7a5-8bb77ac8d684 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action" status: 202 len: 0 microversion: 2.1 time: 0.122858 Sep 18 01:05:34.644003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 432/863] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:05:34 2022] POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action => generated 0 bytes in 124 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:05:34.650007 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=4 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:34.654291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:34.655011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:34.661975 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:34.662477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:34.662915 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:34.663607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:34.664060 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:34.664483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:34.794438 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:34.802621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7416e79c-1cba-4a71-80dc-3f51d856f3ba tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.154014 Sep 18 01:05:34.803182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 432/864] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:34 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:35.825915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:35.826973 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:35.833467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:35.833945 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:35.834375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:35.835076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:35.835520 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:35.836181 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:35.968573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:35.977130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6860ecc6-a409-40f2-b532-2a005d9f0e18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.155390 Sep 18 01:05:35.977687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 433/865] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:35 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:37.001520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:37.002081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:37.009195 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:37.009827 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:37.010295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:37.011093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:37.011464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:37.011886 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:37.154853 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:37.164273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ca9ea96f-3688-42fd-828a-ade6d1d918ed tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.167200 Sep 18 01:05:37.171684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 433/866] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:36 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:37.566541 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1710710319", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1667373744", "security_groups": [{"name": "tempest-secgroup-smoke-786252742"}], "networks": [{"uuid": "30756a9b-3b35-450a-9804-89022f2d94c9"}], "block_device_mapping_v2": [{"uuid": "9b045fb3-5c20-4fd6-95c5-03c8d0871602", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:37.677918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] validate_networks() for [('30756a9b-3b35-450a-9804-89022f2d94c9', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:05:38.051858 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:05:38.052308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:05:38.052772 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:05:38.053176 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:05:38.053581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:05:38.062210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:38.063211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:38.063641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:38.064366 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:38.064801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:38.065217 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:38.082966 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Getting quotas for project 2d812ad5fce24a49b0f2d07081d5718a. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:05:38.087118 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Getting quotas for user 7ea84c1e0dcf441fb1346eaaef4de421 and project 2d812ad5fce24a49b0f2d07081d5718a. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:05:38.145602 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:05:38.147751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='9b045fb3-5c20-4fd6-95c5-03c8d0871602',volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:05:38.192240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:38.193445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:38.203754 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:38.204576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:38.205171 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:38.206389 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:38.206997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:38.207591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:38.285398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b261e347-94e5-4877-8640-b83978da44c1 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.722683 Sep 18 01:05:38.292014 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 434/867] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:05:37 2022] POST /compute/v2.1/servers => generated 405 bytes in 724 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:05:38.299155 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2e191eec-07da-4da8-bc07-c2f1a61435f6 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:38.299881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2e191eec-07da-4da8-bc07-c2f1a61435f6 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:38.369879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2e191eec-07da-4da8-bc07-c2f1a61435f6 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:38.373782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:38.380781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2e191eec-07da-4da8-bc07-c2f1a61435f6 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1089 microversion: 2.1 time: 0.085420 Sep 18 01:05:38.381604 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 435/868] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:38 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1089 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:38.396483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5f96e04f-a143-4453-b880-a90224272cb1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.209304 Sep 18 01:05:38.397451 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 434/869] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:38 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:39.405424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:39.406060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:39.412603 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:39.413084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:39.413516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:39.414198 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:39.414674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:39.415098 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:39.419555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:39.420258 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:39.427291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:39.427774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:39.428212 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:39.428911 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:39.429368 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:39.429794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:39.485214 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:39.492934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-57b201a4-71b4-4d1c-a4c5-5871bae03976 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1139 microversion: 2.1 time: 0.091336 Sep 18 01:05:39.493510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 436/870] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:39 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1139 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:39.570569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:39.578738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-153efa45-d38a-4554-9e44-0bda6d2d7624 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.163080 Sep 18 01:05:39.579304 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 435/871] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:39 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:40.515270 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:40.516446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:40.523215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:40.523711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:40.524150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:40.525073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:40.525835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:40.526747 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:40.593153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:40.601484 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e6937c45-2036-4940-a2fa-c52fb44c6389 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1187 microversion: 2.1 time: 0.090057 Sep 18 01:05:40.602063 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 437/872] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:40 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1187 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:40.603281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:40.603895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:40.610884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:40.611356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:40.611787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:40.612487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:40.612949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:40.613369 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:40.740805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:40.748893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-93acd4f5-4c46-4e57-a3d3-2762f62df85f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.150553 Sep 18 01:05:40.749459 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 436/873] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:40 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:41.624078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:41.624714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:41.631400 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:41.631886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:41.632320 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:41.633016 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:41.633461 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:41.633877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:41.773125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:41.773817 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:41.780430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:41.781009 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:41.781520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:41.782303 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:41.782891 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:41.783422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:41.952238 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:41.960433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-bd485635-4fd8-47d6-b88f-0d7d1c306a1f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1272 microversion: 2.1 time: 0.340125 Sep 18 01:05:41.961012 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 438/874] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:41 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1272 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:42.048772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:42.057337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-af6070cb-3c4f-4294-a5a1-324ec84606ad tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.288555 Sep 18 01:05:42.057917 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 437/875] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:41 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:42.618922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-46c82a62-eaec-471c-8dcb-1a00386b81cd req-cb751b26-1252-4d5c-a0f1-d30636899062 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69223fa4-55c0-4c65-b9d4-c2e5ec0b778d", "tag": "ac27aed2-9401-48f5-a764-f83ca47284b4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:42.629872 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-46c82a62-eaec-471c-8dcb-1a00386b81cd req-cb751b26-1252-4d5c-a0f1-d30636899062 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:42.630690 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-46c82a62-eaec-471c-8dcb-1a00386b81cd req-cb751b26-1252-4d5c-a0f1-d30636899062 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:42.630959 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-46c82a62-eaec-471c-8dcb-1a00386b81cd req-cb751b26-1252-4d5c-a0f1-d30636899062 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:42.642938 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-46c82a62-eaec-471c-8dcb-1a00386b81cd req-cb751b26-1252-4d5c-a0f1-d30636899062 service nova] Creating event network-changed:ac27aed2-9401-48f5-a764-f83ca47284b4 for instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d on hv-cinder-maste Sep 18 01:05:42.647770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-46c82a62-eaec-471c-8dcb-1a00386b81cd req-cb751b26-1252-4d5c-a0f1-d30636899062 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032640 Sep 18 01:05:42.648369 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 439/876] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:05:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:05:42.985222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:42.986016 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:42.992659 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:42.993153 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:42.993586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:42.994286 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:42.994767 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:42.995193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:43.082017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:43.082749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:43.089156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:43.089648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:43.090086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:43.090813 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:43.091269 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:43.091695 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:43.190492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:43.199785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-110b3147-d848-43bf-ad52-73cf0a4c5694 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1272 microversion: 2.1 time: 0.218839 Sep 18 01:05:43.200540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 438/877] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:42 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1272 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:43.304421 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:43.312673 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8eb11191-4e0e-44d2-a708-3255eda21200 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.235322 Sep 18 01:05:43.313243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 440/878] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:43 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:44.223266 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:44.223869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:44.230395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:44.230904 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:44.231337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:44.232016 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:44.232463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:44.232879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:44.337301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:44.338031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:44.344614 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:44.345091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:44.345560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:44.346242 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:44.346724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:44.347128 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:44.368824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:44.376541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d4c49c5e-7a0f-4215-bbe9-ba0fd994576e tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.157130 Sep 18 01:05:44.377611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 439/879] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:44 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:44.472268 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:44.480749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-31e7c52f-222a-4f7f-8c68-be66c8030b8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.148149 Sep 18 01:05:44.481273 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 441/880] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:44 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:44.667253 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-587641433", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-58756093", "security_groups": [{"name": "tempest-secgroup-smoke-821199059"}], "block_device_mapping_v2": [{"uuid": "b4a17b9c-bd3f-4230-8f83-b86be2b9a4df", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "4ff6b087-81f6-47ef-8679-6af29cd7449d"}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:44.777395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] validate_networks() for [('4ff6b087-81f6-47ef-8679-6af29cd7449d', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:05:45.389272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:05:45.389839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:05:45.390144 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:05:45.390579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:05:45.390989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:05:45.400308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:45.400804 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:45.400921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:45.401043 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:45.401259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:45.401989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:45.402444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:45.402895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:45.407281 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:45.407771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:45.408204 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:45.409452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:45.409910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:45.410333 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:45.420196 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for project 6958209222b44905bcb4baa1b8c26c58. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:05:45.424182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Getting quotas for user 823fa0c3eeeb489591c639b81afc02bc and project 6958209222b44905bcb4baa1b8c26c58. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:05:45.483090 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:05:45.485273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b4a17b9c-bd3f-4230-8f83-b86be2b9a4df',volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:05:45.549295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:45.556821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e237169a-4ce6-46a7-ade3-088cb36fe96c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.160281 Sep 18 01:05:45.557394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 442/881] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:45 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:45.572840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:45.572840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:45.572840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:45.572840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:45.573248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:45.573248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:45.573248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:45.573248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:45.628354 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bb2f8a77-eeb3-450e-82fe-ffe00b91abe0 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.964292 Sep 18 01:05:45.629010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 440/882] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:05:44 2022] POST /compute/v2.1/servers => generated 405 bytes in 966 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:05:45.640408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-05a47fd6-8037-4e19-9bcd-9194dd198783 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:45.640929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-05a47fd6-8037-4e19-9bcd-9194dd198783 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:45.715005 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-05a47fd6-8037-4e19-9bcd-9194dd198783 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:45.723057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-05a47fd6-8037-4e19-9bcd-9194dd198783 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1092 microversion: 2.1 time: 0.086500 Sep 18 01:05:45.723643 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 441/883] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:45 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1092 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:45.751343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:45.760827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cf07301e-d801-4fcd-b368-65e7f2fc6dcd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.202399 Sep 18 01:05:45.761475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 443/884] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:45 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:46.581089 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:46.581735 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:46.588890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:46.589367 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:46.589796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:46.590496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:46.590949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:46.591366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:46.721229 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:46.729682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d869a905-2ccd-42cb-99af-da780a7d8116 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.153114 Sep 18 01:05:46.730239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 442/885] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:46 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:46.745141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:46.745758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:46.752565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:46.753061 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:46.753552 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:46.754256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:46.754807 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:46.755256 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:46.783740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:46.784891 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:46.790988 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:46.791469 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:46.791898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:46.792589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:46.793040 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:46.793459 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:46.824826 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:46.834873 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-813118c0-fd32-43ef-ae1e-a6dd5a953698 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1190 microversion: 2.1 time: 0.093206 Sep 18 01:05:46.835624 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 444/886] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:46 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1190 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:47.017093 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:47.025942 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-13629cde-dc2e-4d79-8104-8cb53714a0a9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.246025 Sep 18 01:05:47.026528 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 443/887] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:46 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:47.757526 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:47.758170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:47.765675 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:47.766396 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:47.767045 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:47.767936 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:47.768620 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:47.769207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:47.859010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:47.859573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:47.866769 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:47.867253 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:47.867685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:47.868388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:47.868839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:47.869260 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:48.072472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:48.106058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:48.115083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b13f1181-5896-4be0-8cb9-8057d12c2fab tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.365602 Sep 18 01:05:48.115726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 445/888] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:47 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:48.121971 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:48.122758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:48.129516 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:48.130084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:48.130676 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:48.131850 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:48.132360 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:48.132846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:48.141347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:48.150161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b49d336b-35d4-4e04-bf15-42408055d294 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.295049 Sep 18 01:05:48.151111 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 444/889] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:47 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:48.375710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=8 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:48.387246 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:48.401752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4efd7918-7ba7-4a2f-84eb-4efec1155976 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.283669 Sep 18 01:05:48.402671 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 446/890] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:48 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:48.594918 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-4a07f9aa-cb2c-4f3c-89a3-ef09cb011810 req-96ae39f7-9d45-464e-8687-7f1176c7a44a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c", "tag": "060a9fb2-025c-4f62-be79-f392dbda95a1"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:48.606810 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-4a07f9aa-cb2c-4f3c-89a3-ef09cb011810 req-96ae39f7-9d45-464e-8687-7f1176c7a44a service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:48.607621 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-4a07f9aa-cb2c-4f3c-89a3-ef09cb011810 req-96ae39f7-9d45-464e-8687-7f1176c7a44a service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:48.608065 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-4a07f9aa-cb2c-4f3c-89a3-ef09cb011810 req-96ae39f7-9d45-464e-8687-7f1176c7a44a service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:48.619565 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-4a07f9aa-cb2c-4f3c-89a3-ef09cb011810 req-96ae39f7-9d45-464e-8687-7f1176c7a44a service nova] Creating event network-changed:060a9fb2-025c-4f62-be79-f392dbda95a1 for instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c on hv-cinder-maste Sep 18 01:05:48.624793 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-4a07f9aa-cb2c-4f3c-89a3-ef09cb011810 req-96ae39f7-9d45-464e-8687-7f1176c7a44a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033631 Sep 18 01:05:48.625701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 445/891] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:05:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:05:49.139796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:49.140378 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:49.146795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:49.147407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:49.147845 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:49.148531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:49.148983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:49.149405 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:49.178575 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:49.179341 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:49.185538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:49.186022 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:49.186477 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:49.187187 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:49.187641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:49.188066 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:49.330564 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:49.338466 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4e7568e2-9c76-4c48-b5bb-1a46fa186418 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.203080 Sep 18 01:05:49.339048 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 447/892] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:49 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:49.360999 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:49.369273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6c221040-a9fc-4026-ae41-1aa34ce7691c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.195820 Sep 18 01:05:49.369832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 446/893] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:49 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:49.433021 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:49.433550 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:49.439595 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:49.440074 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:49.440506 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:49.441199 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:49.441650 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:49.442076 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:49.597947 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:49.606088 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6ff5d0ae-83e9-4eae-87d1-f379c19cadc7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.177103 Sep 18 01:05:49.606684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 448/894] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:49 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:50.362424 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:50.363658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:50.366088 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=4 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:50.371356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:50.371841 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:50.372272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:50.372973 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:50.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:50.373837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:50.377222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:50.392493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:50.393112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:50.399533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:50.400298 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:50.400888 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:50.401757 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:50.402362 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:50.402971 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:50.686402 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:50.690028 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:50.692460 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:50.696934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ab717e53-ebbc-41b8-aef6-c069f99a8897 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.308097 Sep 18 01:05:50.710490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 449/895] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:50 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:50.710490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:50.710490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:50.716781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:50.717134 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-86bb9796-3dd3-4261-b847-16c7f82921d7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.358966 Sep 18 01:05:50.717284 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:50.717753 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 447/896] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:50 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:50.725009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:50.725009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:50.725009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:50.725009 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:50.905250 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:50.914306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9d1c5a42-80d3-412b-a249-c07c27e298fd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.215555 Sep 18 01:05:50.914923 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 450/897] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:50 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:51.724797 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:51.725446 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:51.732060 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:51.732648 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:51.733160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:51.733985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:51.734490 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:51.734903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:51.741505 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:51.742161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:51.749175 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:51.749736 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:51.750241 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:51.751058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:51.751661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:51.752183 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:51.932036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:51.936311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:51.939957 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-aa79fc6c-1b54-4e34-907e-3ddc40985f58 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.220047 Sep 18 01:05:51.941123 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 448/898] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:51 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:51.944807 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-37f79c16-459d-4db4-8df4-c7a012a75372 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.208010 Sep 18 01:05:51.945523 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 451/899] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:51 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:51.945653 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:51.946131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:51.951897 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:51.952378 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:51.952808 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:51.953498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:51.953951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:51.954375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:52.083952 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:52.092400 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fd4090f6-8646-4e2a-be92-99876d5237a4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.150266 Sep 18 01:05:52.092889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 449/900] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:51 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:52.963926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:52.964544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:52.967393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:52.968012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:52.971215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:52.971724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:52.972164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:52.972870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:52.973324 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:52.973749 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:52.975184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:52.975681 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:52.976120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:52.976826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:52.977282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:52.977712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:53.172694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:53.182081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2c3e6ac7-03ff-40e7-872b-fde50b21acca tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.222214 Sep 18 01:05:53.182711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 452/901] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:52 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:53.185875 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:53.187942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:53.188803 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:53.195475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:53.195629 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c2c5599b-abd3-4a20-9c93-8ff045ccd995 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.232059 Sep 18 01:05:53.196326 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:53.197126 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:53.198056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:53.198612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:53.199102 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:53.207160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 450/902] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:52 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:53.336186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:53.344657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cbed6c8b-e611-4a24-b5ac-f7af28fffe6d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.160689 Sep 18 01:05:53.345230 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 453/903] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:53 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:54.205197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:54.206339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:54.213255 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:54.213746 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:54.214189 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:54.215325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:54.215785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:54.216216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:54.219124 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:54.219736 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:54.226831 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:54.227543 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:54.228150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:54.229028 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:54.229640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:54.230214 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:54.462727 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:54.467405 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:54.471094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d5c486b2-7a0a-4879-b8ef-b97818bdb344 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.269785 Sep 18 01:05:54.471709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 451/904] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:54 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:54.475490 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-af6afc07-f2cc-436a-8a85-8f91e92b3718 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.260136 Sep 18 01:05:54.476295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 454/905] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:54 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:54.476423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:54.477019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:54.482789 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:54.483272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:54.483707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:54.484395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:54.484877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:54.485299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:54.620087 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:54.628916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-24ed2974-4fdc-4fbe-84e4-bb3e12d8e23e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1722 microversion: 2.1 time: 0.156248 Sep 18 01:05:54.629484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 452/906] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:54 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:55.260117 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-57ff3bce-1cc8-4914-9409-01a19ee5c871 req-205f6ce8-1f5e-4457-bde5-736e6b7daf2b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69223fa4-55c0-4c65-b9d4-c2e5ec0b778d", "name": "network-vif-plugged", "status": "completed", "tag": "ac27aed2-9401-48f5-a764-f83ca47284b4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:55.272067 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-57ff3bce-1cc8-4914-9409-01a19ee5c871 req-205f6ce8-1f5e-4457-bde5-736e6b7daf2b service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:55.272660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-57ff3bce-1cc8-4914-9409-01a19ee5c871 req-205f6ce8-1f5e-4457-bde5-736e6b7daf2b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:55.273162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-57ff3bce-1cc8-4914-9409-01a19ee5c871 req-205f6ce8-1f5e-4457-bde5-736e6b7daf2b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:55.286925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-57ff3bce-1cc8-4914-9409-01a19ee5c871 req-205f6ce8-1f5e-4457-bde5-736e6b7daf2b service nova] Creating event network-vif-plugged:ac27aed2-9401-48f5-a764-f83ca47284b4 for instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d on hv-cinder-maste Sep 18 01:05:55.291854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-57ff3bce-1cc8-4914-9409-01a19ee5c871 req-205f6ce8-1f5e-4457-bde5-736e6b7daf2b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035969 Sep 18 01:05:55.292884 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 455/907] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:05:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:05:55.493664 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:55.494274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:55.496620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:55.499789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:55.500424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:55.502039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:55.502556 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:55.502991 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:55.503689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:55.504141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:55.504562 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:55.506827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:55.507310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:55.507738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:55.508441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:55.508943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:55.509397 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:55.660203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:55.668026 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-90b5f89d-8175-4a2c-a1c9-7292b07baf08 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.172652 Sep 18 01:05:55.668598 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 456/908] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:55 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:55.673072 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:55.673854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:55.679775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:55.680262 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:55.680694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:55.681381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:55.681828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:55.682247 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:55.720614 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=8 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:05:56.116159 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:56.123879 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f7faefb3-fa9b-42dd-9730-40b16fe393dd tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.634052 Sep 18 01:05:56.124581 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 453/909] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:55 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:56.131985 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:56.140847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-62a39c09-f949-44fb-ac78-66566162bcce tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1699 microversion: 2.1 time: 0.471205 Sep 18 01:05:56.141440 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 457/910] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:55 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1699 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:56.166282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a9930667-772a-4fef-89ca-38acf4e28b1b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:05:56.174038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9930667-772a-4fef-89ca-38acf4e28b1b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:56.175268 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9930667-772a-4fef-89ca-38acf4e28b1b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:56.175812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a9930667-772a-4fef-89ca-38acf4e28b1b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:56.237585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a9930667-772a-4fef-89ca-38acf4e28b1b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Going to try to start instance {{(pid=120832) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 18 01:05:56.280240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a9930667-772a-4fef-89ca-38acf4e28b1b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action" status: 202 len: 0 microversion: 2.1 time: 0.118660 Sep 18 01:05:56.280824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 454/911] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:05:56 2022] POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:05:56.290203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:56.290937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:56.298006 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:56.298519 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:56.298952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:56.300284 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:56.300738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:56.301156 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:56.428662 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:56.436774 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f1453755-7d29-4884-aa9f-8e0fcaa8f770 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1708 microversion: 2.1 time: 0.151284 Sep 18 01:05:56.437346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 458/912] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:56 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1708 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:56.692605 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:56.693300 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:56.699861 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:56.700339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:56.700774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:56.701462 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:56.701916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:56.702337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:56.833174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:56.841421 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2e219fe2-66c5-4d61-9632-1c2fd1f588d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1260 microversion: 2.1 time: 0.153579 Sep 18 01:05:56.841979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 455/913] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:56 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:57.207000 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:57.207605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:57.213745 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:57.214227 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:57.214689 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:57.215379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:57.215829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:57.216248 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:57.347515 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:57.355159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-09a28aa6-e254-44ab-98d1-42a7f0a82af8 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.152028 Sep 18 01:05:57.355711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 459/914] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:57 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:57.459913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:57.460521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:57.466821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:57.467305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:57.467732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:57.468435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:57.468882 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:57.469289 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:57.677537 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:57.685832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8922f4c0-8902-4a40-b39f-4c3e64b97be7 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1708 microversion: 2.1 time: 0.229891 Sep 18 01:05:57.686404 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 456/915] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:57 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1708 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:05:57.863953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:57.864560 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:57.870752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:57.871234 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:57.871672 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:57.872355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:57.872808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:57.873231 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:58.002059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:58.009830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-cdb5133a-85d9-42b2-8749-4758aaa729c7 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1444 microversion: 2.1 time: 0.149760 Sep 18 01:05:58.010401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 460/916] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:57 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1444 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:58.032191 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:58.032799 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:58.040061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:58.040539 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:58.040971 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:58.041655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:58.042100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:58.042547 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:58.186710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:58.194869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-87b8288f-67e2-4b5e-a0e7-e72f4ab5a970 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1444 microversion: 2.1 time: 0.166482 Sep 18 01:05:58.195425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 457/917] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:58 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1444 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:58.378237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:58.378980 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:58.385687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:58.386177 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:58.386652 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:58.387350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:58.387808 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:58.388237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:58.524384 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:58.532099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-18c8a10f-4e31-4e98-aa7c-4b7e8dcbc434 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.158579 Sep 18 01:05:58.532791 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 461/918] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:58 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:58.708899 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:58.709541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:58.716050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:58.716458 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:58.716891 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:58.717595 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:58.718044 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:58.718481 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:58.859791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:58.868641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-34fa6eb5-334f-4612-abf2-226eda21d6b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1708 microversion: 2.1 time: 0.163568 Sep 18 01:05:58.869226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 458/919] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:58 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1708 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:59.554815 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:59.555438 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:59.561961 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:59.562486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:59.562925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:59.563624 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:59.564080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:59.564513 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:59.705401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:05:59.717766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2909808f-ed33-489f-8298-c1595029fa9a tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.166892 Sep 18 01:05:59.719398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 462/920] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:59 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:05:59.893435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:05:59.894051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:05:59.900644 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:59.901118 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:59.901559 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:05:59.902257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:05:59.902775 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:05:59.903320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:00.031483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:00.040458 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-dc14a6e6-ed3b-424f-8154-2a6e50748fde tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1712 microversion: 2.1 time: 0.151222 Sep 18 01:06:00.040956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 459/921] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:05:59 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1712 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:00.150330 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-eee18c7c-866c-4f85-8ec6-f8a8100acd47 req-7bc32ccc-fa7a-43c3-bbd2-e182b14cfb5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69223fa4-55c0-4c65-b9d4-c2e5ec0b778d", "tag": "ac27aed2-9401-48f5-a764-f83ca47284b4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:06:00.160589 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-eee18c7c-866c-4f85-8ec6-f8a8100acd47 req-7bc32ccc-fa7a-43c3-bbd2-e182b14cfb5e service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:00.161152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-eee18c7c-866c-4f85-8ec6-f8a8100acd47 req-7bc32ccc-fa7a-43c3-bbd2-e182b14cfb5e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:00.161654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-eee18c7c-866c-4f85-8ec6-f8a8100acd47 req-7bc32ccc-fa7a-43c3-bbd2-e182b14cfb5e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:00.174311 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-eee18c7c-866c-4f85-8ec6-f8a8100acd47 req-7bc32ccc-fa7a-43c3-bbd2-e182b14cfb5e service nova] Creating event network-changed:ac27aed2-9401-48f5-a764-f83ca47284b4 for instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d on hv-cinder-maste Sep 18 01:06:00.179356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-eee18c7c-866c-4f85-8ec6-f8a8100acd47 req-7bc32ccc-fa7a-43c3-bbd2-e182b14cfb5e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032862 Sep 18 01:06:00.180101 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 463/922] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:06:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:06:00.741401 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:00.742020 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:00.748101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:00.748590 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:00.749023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:00.749723 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:00.750174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:00.750624 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:00.888837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:00.896834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7e2ca39e-c847-452c-8cdf-e0042e5d6dfb tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.159451 Sep 18 01:06:00.898110 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 460/923] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:00 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:01.351827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-75a39d93-c098-4e85-8cff-49413481db6a req-2e965ed3-2a22-4b58-b422-391d68ff71e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c", "name": "network-vif-plugged", "status": "completed", "tag": "060a9fb2-025c-4f62-be79-f392dbda95a1"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:06:01.361711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-75a39d93-c098-4e85-8cff-49413481db6a req-2e965ed3-2a22-4b58-b422-391d68ff71e5 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:01.362894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-75a39d93-c098-4e85-8cff-49413481db6a req-2e965ed3-2a22-4b58-b422-391d68ff71e5 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:01.363370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-75a39d93-c098-4e85-8cff-49413481db6a req-2e965ed3-2a22-4b58-b422-391d68ff71e5 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:01.527258 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-75a39d93-c098-4e85-8cff-49413481db6a req-2e965ed3-2a22-4b58-b422-391d68ff71e5 service nova] Creating event network-vif-plugged:060a9fb2-025c-4f62-be79-f392dbda95a1 for instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c on hv-cinder-maste Sep 18 01:06:01.532043 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-75a39d93-c098-4e85-8cff-49413481db6a req-2e965ed3-2a22-4b58-b422-391d68ff71e5 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.185705 Sep 18 01:06:01.532660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 464/924] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:06:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:06:01.920648 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:01.921260 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:01.927671 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:01.928148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:01.928582 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:01.929271 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:01.929724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:01.930148 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:02.062914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:02.071151 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6c0e703a-bbdd-4064-a777-dc34e04a656c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.154421 Sep 18 01:06:02.071711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 461/925] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:01 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:03.095556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:03.096253 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:03.103231 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:03.103717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:03.104154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:03.104847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:03.105303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:03.105732 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:03.238347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:03.245989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-da568e49-d97c-4309-a6b9-391a337546de tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1263 microversion: 2.1 time: 0.155285 Sep 18 01:06:03.246565 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 465/926] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:03 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1263 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:04.268971 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:04.269685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:04.275646 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:04.276128 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:04.276563 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:04.277247 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:04.277699 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:04.278120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:04.411494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:04.413778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:04.421426 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-87ad6f9e-f673-4fe8-8a10-12dc7e62b8e4 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1451 microversion: 2.1 time: 0.156495 Sep 18 01:06:04.422000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 462/927] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:04 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1451 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:04.444467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:04.445082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:04.451602 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:04.452090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:04.452529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:04.453226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:04.453684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:04.454119 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:04.599995 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:04.607852 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e3ee5740-473c-4694-9d51-e1fd8f182443 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1451 microversion: 2.1 time: 0.167280 Sep 18 01:06:04.608414 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 466/928] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:04 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1451 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:06.165973 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-860d0b44-ba5f-4328-b7c1-6a6beb1997e6 req-e05dc124-3533-4a61-92c1-8714c331e806 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c", "tag": "060a9fb2-025c-4f62-be79-f392dbda95a1"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:06:06.177216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-860d0b44-ba5f-4328-b7c1-6a6beb1997e6 req-e05dc124-3533-4a61-92c1-8714c331e806 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:06.177961 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-860d0b44-ba5f-4328-b7c1-6a6beb1997e6 req-e05dc124-3533-4a61-92c1-8714c331e806 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:06.178390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-860d0b44-ba5f-4328-b7c1-6a6beb1997e6 req-e05dc124-3533-4a61-92c1-8714c331e806 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:06.189453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-860d0b44-ba5f-4328-b7c1-6a6beb1997e6 req-e05dc124-3533-4a61-92c1-8714c331e806 service nova] Creating event network-changed:060a9fb2-025c-4f62-be79-f392dbda95a1 for instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c on hv-cinder-maste Sep 18 01:06:06.194622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-860d0b44-ba5f-4328-b7c1-6a6beb1997e6 req-e05dc124-3533-4a61-92c1-8714c331e806 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032825 Sep 18 01:06:06.195309 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 463/929] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:06:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:06:38.724352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:38.725514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:38.727718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=3 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:38.728375 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=90 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:38.734660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:38.735141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:38.735573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:38.736351 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:38.736825 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:38.737227 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:38.817581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lazy-loading 'services' on Instance uuid 529a4d99-a333-4068-a419-09d67a1ae170 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:06:38.896017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e13e1177-5fd7-4084-86f2-7a295dfaa228 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-volume_attachments/5cc7c857-43b1-423f-ab8e-6cb3d2f2a9d2" status: 202 len: 0 microversion: 2.1 time: 0.175595 Sep 18 01:06:38.896743 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 467/930] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:06:38 2022] DELETE /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-volume_attachments/5cc7c857-43b1-423f-ab8e-6cb3d2f2a9d2 => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:06:40.014294 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=33 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:40.015083 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:06:40.019504 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-eea53940-c872-40b2-8026-3ac746491f43 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:06:40.027018 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eea53940-c872-40b2-8026-3ac746491f43 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:40.027509 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eea53940-c872-40b2-8026-3ac746491f43 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:40.027943 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eea53940-c872-40b2-8026-3ac746491f43 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:40.050064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-eea53940-c872-40b2-8026-3ac746491f43 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Going to try to stop instance {{(pid=120832) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 18 01:06:40.091845 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-eea53940-c872-40b2-8026-3ac746491f43 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action" status: 202 len: 0 microversion: 2.1 time: 0.079107 Sep 18 01:06:40.092457 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 464/931] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:06:40 2022] POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action => generated 0 bytes in 81 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:06:40.102179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:40.103496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:40.110144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:40.110706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:40.111141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:40.111838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:40.112283 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:40.112702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:40.240886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:40.248953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-aa032c7a-f40e-4fe5-8caf-4a8227f5dbfd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.152049 Sep 18 01:06:40.249527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 468/932] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:40 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:41.275660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:41.276376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:41.282968 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:41.283453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:41.283888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:41.284580 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:41.285036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:41.285455 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:41.418165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:41.425784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-067a8076-6ce2-4c10-ad6d-c4269f835e42 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.156819 Sep 18 01:06:41.426347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 465/933] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:41 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:42.448455 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:42.449058 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:42.455612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:42.456097 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:42.456533 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:42.457225 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:42.457677 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:42.458099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:42.584651 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:42.592472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8a2667f1-157f-4d66-a5f9-ac7dc46ca64a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.147887 Sep 18 01:06:42.593033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 469/934] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:42 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:43.616396 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:43.617006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:43.623433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:43.623913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:43.624346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:43.625032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:43.625484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:43.625906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:43.752877 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:43.761173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f3fe0a11-3536-477a-a993-ace422a36290 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.149165 Sep 18 01:06:43.761740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 466/935] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:43 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:44.784324 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:44.784878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:44.791379 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:44.791857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:44.792296 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:44.792984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:44.793434 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:44.793857 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:44.925140 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:44.932835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f5383382-868c-4fe3-9428-a6ceafe2f690 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.152619 Sep 18 01:06:44.933401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 470/936] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:44 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:45.956704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:45.957344 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:45.965909 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:45.966388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:45.966841 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:45.967527 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:45.967976 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:45.968394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:46.094240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:46.102478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d598509f-1ea6-4418-826a-59c4c85aea79 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.149875 Sep 18 01:06:46.103120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 467/937] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:45 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:47.125081 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:47.125684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:47.132324 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:47.132934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:47.133412 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:47.134150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:47.134625 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:47.135046 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:47.263498 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:47.271184 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-234e4659-6bfa-45a3-b8c2-029af07e4938 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.149986 Sep 18 01:06:47.272290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 471/938] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:47 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:48.294940 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:48.295544 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:48.301890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:48.302371 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:48.303362 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:48.304107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:48.304556 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:48.304999 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:48.438752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:48.446357 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fc49cd85-9b74-4727-b2c8-da82113089a5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.155295 Sep 18 01:06:48.446953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 468/939] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:48 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:49.469638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:49.470265 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:49.476734 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:49.477210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:49.477639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:49.478321 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:49.478823 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:49.479214 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:49.610114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:49.618479 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-bddc13be-51ef-4749-900c-0161f6446f2a tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1676 microversion: 2.1 time: 0.152814 Sep 18 01:06:49.619036 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 472/940] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:49 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1676 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:50.641155 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:50.641703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:50.648833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:50.649306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:50.649740 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:50.650425 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:50.650909 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:50.651324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:50.839167 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:50.846811 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d42cd6a2-c295-4aad-9a6f-10b4e4691bc3 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1653 microversion: 2.1 time: 0.209593 Sep 18 01:06:50.847377 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 469/941] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:50 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1653 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:50.868804 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a20be65d-a3d5-4e00-91fe-b54072c8895f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:06:50.875232 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a20be65d-a3d5-4e00-91fe-b54072c8895f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:50.875721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a20be65d-a3d5-4e00-91fe-b54072c8895f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:50.876165 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a20be65d-a3d5-4e00-91fe-b54072c8895f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:50.898789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a20be65d-a3d5-4e00-91fe-b54072c8895f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Going to try to start instance {{(pid=120831) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 18 01:06:50.939005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a20be65d-a3d5-4e00-91fe-b54072c8895f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action" status: 202 len: 0 microversion: 2.1 time: 0.073827 Sep 18 01:06:50.939531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 473/942] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:06:50 2022] POST /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/action => generated 0 bytes in 75 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:06:50.948170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:50.948786 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:50.955054 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:50.955599 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:50.956272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:50.957146 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:50.957677 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:50.958160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:51.094321 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:51.102472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2cfa46df-8948-4b19-b465-9ddbc3caf8b8 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1662 microversion: 2.1 time: 0.158152 Sep 18 01:06:51.103071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 470/943] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:50 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1662 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:52.126738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:52.127433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:52.133695 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:52.134164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:52.134642 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:52.135332 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:52.135788 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:52.136210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:52.265941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:52.273584 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-eef186d1-f6a4-4fe7-8c89-f3cd9281f2c2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1662 microversion: 2.1 time: 0.151522 Sep 18 01:06:52.274144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 474/944] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:52 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1662 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:06:53.297487 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:06:53.298087 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:06:53.304944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:53.305420 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:53.305853 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:53.306559 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:06:53.307010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:06:53.307429 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:06:53.434924 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:06:53.442577 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e8f8862a-789e-42d0-ba14-9c01c8085a8e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1666 microversion: 2.1 time: 0.149190 Sep 18 01:06:53.443135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 471/945] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:06:53 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1666 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:26.238721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:26.241454 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:26.243627 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=16 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:26.244280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=55,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:26.245430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 01:07:26.245664 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Sun Sep 18 01:07:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-volume_attachments (ip 10.222.0.42) !!! Sep 18 01:07:26.259031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:26.259517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:26.259954 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:26.260647 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:26.261103 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:26.261557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:26.291929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9aecf9f9-a165-4fc2-b44f-a084cddfa065 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.057288 Sep 18 01:07:26.292482 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 475/946] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:07:26 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170/os-volume_attachments => generated 25 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:07:26.356023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:26.356551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:26.358823 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=16 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:26.359473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:26.366129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:26.366706 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:26.367141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:26.367824 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:26.368271 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:26.368689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:26.395448 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:07:26.471135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3a9ebe0b-e8e1-44cd-9d0d-425866d2c169 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 204 len: 0 microversion: 2.1 time: 0.118460 Sep 18 01:07:26.471638 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 472/947] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:07:26 2022] DELETE /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:07:26.479809 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:26.480422 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:26.486330 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:26.486840 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:26.487274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:26.487956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:26.488407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:26.488829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:26.674479 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:26.682846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e1b7aaa1-76a8-4038-8f7d-02a464655900 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1672 microversion: 2.1 time: 0.206866 Sep 18 01:07:26.683413 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 476/948] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:26 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1672 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:27.706161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:27.706953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:27.713929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:27.714408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:27.715125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:27.715821 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:27.716264 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:27.716683 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:27.845083 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:27.852847 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5f2cd889-6bf5-4965-b1a1-237341f992cf tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1672 microversion: 2.1 time: 0.150896 Sep 18 01:07:27.853407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 473/949] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:27 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1672 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:28.876121 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:28.876752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:28.884025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:28.884501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:28.884928 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:28.885811 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:28.886268 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:28.886728 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:29.104770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:29.112430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-31dbb6d8-ba1c-485b-8d78-165d1ebf4b26 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1672 microversion: 2.1 time: 0.240168 Sep 18 01:07:29.112987 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 477/950] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:28 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1672 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:29.727517 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-7dcd7f10-b5ac-4f0c-8376-0239e9776c75 req-573d5afb-1133-4343-bc3d-b0e766ca86be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "529a4d99-a333-4068-a419-09d67a1ae170", "name": "network-vif-deleted", "tag": "8a184e82-50ee-43b7-b12b-caa0631860c8"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:29.740796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-7dcd7f10-b5ac-4f0c-8376-0239e9776c75 req-573d5afb-1133-4343-bc3d-b0e766ca86be service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:29.741274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-7dcd7f10-b5ac-4f0c-8376-0239e9776c75 req-573d5afb-1133-4343-bc3d-b0e766ca86be service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:29.741702 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-7dcd7f10-b5ac-4f0c-8376-0239e9776c75 req-573d5afb-1133-4343-bc3d-b0e766ca86be service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:29.753469 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-7dcd7f10-b5ac-4f0c-8376-0239e9776c75 req-573d5afb-1133-4343-bc3d-b0e766ca86be service nova] Creating event network-vif-deleted:8a184e82-50ee-43b7-b12b-caa0631860c8 for instance 529a4d99-a333-4068-a419-09d67a1ae170 on hv-cinder-maste Sep 18 01:07:29.758757 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-7dcd7f10-b5ac-4f0c-8376-0239e9776c75 req-573d5afb-1133-4343-bc3d-b0e766ca86be service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035417 Sep 18 01:07:29.759782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 474/951] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:30.137896 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:30.138539 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:30.145018 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:30.145537 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:30.145982 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:30.147174 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:30.147629 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:30.148180 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:30.179185 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:30.179673 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:30.180108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:30.180581 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lazy-loading 'fault' on Instance uuid 529a4d99-a333-4068-a419-09d67a1ae170 {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:07:30.223499 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:30.231681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1832dcd5-3a8e-400b-9e6b-d7cbd2e483d9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 200 len: 1328 microversion: 2.1 time: 0.098103 Sep 18 01:07:30.232250 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 478/952] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:30 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 1328 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:31.255424 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:31.256186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:31.262534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:31.263026 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:31.263464 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:31.264188 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:31.264665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:31.265091 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:31.278351 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] HTTP exception thrown: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Request method failure captured: Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 HTTP/1.1 Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm6hL0Uvh_1oiIthReAaa8UVyGZ00HAmRKJVSA9KcTBMYrkxbCudVVKZt_k1amCBU1w7HbwCfq9E2yzKMEGCAxyV3xbUCcyh-bovIhA-5gNbBMCKf6fK9mnUu3NDOx0crgBOqdQLV9VT8x1QxI6U_u4rXOadhkMkBmFBSg4o9MkYGBo Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:07:31.280041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}] Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: bd2fa2deb0a24aff9d8b7ce97b6e461b Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:07:31.281482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '529a4d99-a333-4068-a419-09d67a1ae170'} Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:07:31.282415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:07:31.283857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:31.285261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Returning 404 to user: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:07:31.285261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-52c02e61-a669-4ba8-b31e-3e987bab0a91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 404 len: 111 microversion: 2.1 time: 0.030739 Sep 18 01:07:31.285261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 475/953] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:31 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:07:31.291035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f01b487a-2652-420a-97b7-4d1e6bb97cbe tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '.version_select at 0x7fcab1a25b80>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:31.313891 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f01b487a-2652-420a-97b7-4d1e6bb97cbe tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1281574759" status: 202 len: 0 microversion: 2.1 time: 0.027087 Sep 18 01:07:31.314474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 479/954] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Sep 18 01:07:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1281574759 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:07:31.930071 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-18a9dd36-a898-41b0-8573-5279b149e1dc tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'create', calling method: .version_select at 0x7fcab1b12670>, body: {"keypair": {"name": "tempest-keypair-356981727", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAbI3KD+xtT4WrhoqLAuui8p70Mrl2jVrr0Kzm4gFeQM4QrDsvFbfgGM9bl4/Sn9C/gv1XRixR7dAPvi36nDQ97hMSIPoE7Zlkk8WaPDZVFRdt537pINl7oNBoRuqWNrmg=="}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:31.945552 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-18a9dd36-a898-41b0-8573-5279b149e1dc tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:07:31.949436 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-18a9dd36-a898-41b0-8573-5279b149e1dc tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for user bd2fa2deb0a24aff9d8b7ce97b6e461b and project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'key_pairs'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:07:31.971965 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-18a9dd36-a898-41b0-8573-5279b149e1dc tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.045787 Sep 18 01:07:31.972554 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 476/955] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 18 01:07:31 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:33.225300 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-2030563984", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5e1cdc84-fae8-4064-bacf-43638170a453"}], "security_groups": [{"name": "tempest-securitygroup--1586693634"}], "key_name": "tempest-keypair-356981727", "user_data": "}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:33.325452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] validate_networks() for [('5e1cdc84-fae8-4064-bacf-43638170a453', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:07:33.697694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:07:33.698114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:07:33.698628 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:07:33.699034 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:07:33.699439 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:07:33.708083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:33.708553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:33.709635 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:33.710355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:33.710827 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:33.711246 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:33.728365 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:07:33.732661 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Getting quotas for user bd2fa2deb0a24aff9d8b7ce97b6e461b and project 39509a7e0ca64f94913b5fbf10c6ec9c. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:07:33.744002 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:07:33.746289 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:07:33.793343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-b58946cd-061e-400d-b3f8-647b71454ea1 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.573596 Sep 18 01:07:33.794004 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 480/956] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:07:33 2022] POST /compute/v2.1/servers => generated 402 bytes in 575 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:07:33.806023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4dac1350-0e9c-4681-bec4-9681c9ef3f1f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:33.806813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4dac1350-0e9c-4681-bec4-9681c9ef3f1f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:33.861713 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4dac1350-0e9c-4681-bec4-9681c9ef3f1f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:33.869703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4dac1350-0e9c-4681-bec4-9681c9ef3f1f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1237 microversion: 2.1 time: 0.068614 Sep 18 01:07:33.870263 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 477/957] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:33 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1237 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:34.893344 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:34.893934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:34.900606 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:34.901085 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:34.901509 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:34.902188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:34.902667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:34.903091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:34.966738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:34.974286 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c0e970bc-4959-4d8d-9af9-00361da410be tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1289 microversion: 2.1 time: 0.085390 Sep 18 01:07:34.974874 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 481/958] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:34 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1289 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:35.998551 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:35.999143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:36.005615 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:36.006245 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:36.006710 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:36.007410 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:36.007859 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:36.008282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:36.248850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:36.256442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-873e9b0e-bc9e-4f8c-8f41-20989d62cba2 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1363 microversion: 2.1 time: 0.263103 Sep 18 01:07:36.257261 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 478/959] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:35 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1363 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:36.649709 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-6225f974-e684-4d63-89c2-bc51dc52e7f6 req-4798e77a-08d9-448c-b676-0270dd22343b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc403fe0-bf54-4ff8-b714-0cc3f0484d1e", "tag": "bcd94803-b2f3-4f1f-b0ef-275773a5103b"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:36.660544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-6225f974-e684-4d63-89c2-bc51dc52e7f6 req-4798e77a-08d9-448c-b676-0270dd22343b service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:36.660998 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-6225f974-e684-4d63-89c2-bc51dc52e7f6 req-4798e77a-08d9-448c-b676-0270dd22343b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:36.661430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-6225f974-e684-4d63-89c2-bc51dc52e7f6 req-4798e77a-08d9-448c-b676-0270dd22343b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:36.672670 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-6225f974-e684-4d63-89c2-bc51dc52e7f6 req-4798e77a-08d9-448c-b676-0270dd22343b service nova] Creating event network-changed:bcd94803-b2f3-4f1f-b0ef-275773a5103b for instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e on hv-cinder-maste Sep 18 01:07:36.677494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-6225f974-e684-4d63-89c2-bc51dc52e7f6 req-4798e77a-08d9-448c-b676-0270dd22343b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032084 Sep 18 01:07:36.678078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 482/960] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:37.280240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:37.280851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:37.288019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:37.288505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:37.288937 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:37.289628 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:37.290085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:37.290538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:37.453836 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:37.463126 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-be945239-a155-4ed2-b740-f2fa86638475 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1363 microversion: 2.1 time: 0.186791 Sep 18 01:07:37.464364 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 479/961] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:37 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1363 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:38.488527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:38.489290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:38.496137 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:38.496622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:38.497052 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:38.497746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:38.498194 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:38.498687 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:38.708658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:38.716772 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ffffe89c-47d6-48d3-97fc-89bc22af30ec tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1363 microversion: 2.1 time: 0.233687 Sep 18 01:07:38.717345 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 483/962] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:38 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1363 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:39.740623 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:39.741230 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:39.747684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:39.748144 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:39.748571 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:39.749259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:39.749791 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:39.750124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:39.883800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:39.891414 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-27e481da-3de8-4fdd-8e3f-e77fae0c30dd tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1363 microversion: 2.1 time: 0.155190 Sep 18 01:07:39.891992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 480/963] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:39 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:40.914059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:40.914694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:40.921112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:40.921607 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:40.922043 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:40.922759 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:40.923223 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:40.923649 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:41.055332 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:41.063111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-428962c5-0bc3-489e-a7c9-da51daaaff4d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1363 microversion: 2.1 time: 0.152930 Sep 18 01:07:41.063671 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 484/964] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:40 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1363 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:41.864700 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-146cb43c-1530-41d6-bf41-2d1c0ac0b606 req-55f1c5e0-0100-4be1-85b9-27aa495c061c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc403fe0-bf54-4ff8-b714-0cc3f0484d1e", "name": "network-vif-plugged", "status": "completed", "tag": "bcd94803-b2f3-4f1f-b0ef-275773a5103b"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:41.876145 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-146cb43c-1530-41d6-bf41-2d1c0ac0b606 req-55f1c5e0-0100-4be1-85b9-27aa495c061c service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:41.876627 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-146cb43c-1530-41d6-bf41-2d1c0ac0b606 req-55f1c5e0-0100-4be1-85b9-27aa495c061c service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:41.877631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-146cb43c-1530-41d6-bf41-2d1c0ac0b606 req-55f1c5e0-0100-4be1-85b9-27aa495c061c service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:41.889804 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-146cb43c-1530-41d6-bf41-2d1c0ac0b606 req-55f1c5e0-0100-4be1-85b9-27aa495c061c service nova] Creating event network-vif-plugged:bcd94803-b2f3-4f1f-b0ef-275773a5103b for instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e on hv-cinder-maste Sep 18 01:07:41.894833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-146cb43c-1530-41d6-bf41-2d1c0ac0b606 req-55f1c5e0-0100-4be1-85b9-27aa495c061c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033815 Sep 18 01:07:41.895413 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 481/965] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:42.085738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:42.086302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:42.093346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:42.093841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:42.094279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:42.095033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:42.095489 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:42.095907 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:42.249377 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:42.256887 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-df0ab9eb-3393-4f61-9571-5863e5403394 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1363 microversion: 2.1 time: 0.175632 Sep 18 01:07:42.257449 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 485/966] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:42 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1363 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:43.280498 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:43.281197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:43.288530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:43.289013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:43.289445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:43.290130 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:43.290618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:43.291044 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:43.423067 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:43.431340 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d93f71e4-d22c-42a0-8396-f6d010263b75 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1363 microversion: 2.1 time: 0.155049 Sep 18 01:07:43.431906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 482/967] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:43 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1363 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:44.454173 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:44.454830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:44.457356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:44.458029 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=6 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:44.464509 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:44.465403 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:44.465845 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:44.466563 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:44.467033 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:44.467452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:44.595942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:44.604133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ad6fb76e-b0ca-4956-854e-8d03be2b5865 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1549 microversion: 2.1 time: 0.153856 Sep 18 01:07:44.604699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 486/968] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:44 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1549 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:44.626992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:44.627607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:44.629660 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:44.635549 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:44.636027 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:44.636459 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:44.637138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:44.637587 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:44.638008 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:44.715894 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b533117d-613e-44b5-b432-c4d5b2356658 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.093266 Sep 18 01:07:44.716516 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 483/969] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 18 01:07:44 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:45.715578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-12e21140-1ac5-4b5b-9ad6-356b99b8d070 req-fa557d57-ce0d-4a1a-8dd0-649d822a3fe7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc403fe0-bf54-4ff8-b714-0cc3f0484d1e", "tag": "bcd94803-b2f3-4f1f-b0ef-275773a5103b"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:45.726278 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-12e21140-1ac5-4b5b-9ad6-356b99b8d070 req-fa557d57-ce0d-4a1a-8dd0-649d822a3fe7 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:45.726952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-12e21140-1ac5-4b5b-9ad6-356b99b8d070 req-fa557d57-ce0d-4a1a-8dd0-649d822a3fe7 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:45.727493 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-12e21140-1ac5-4b5b-9ad6-356b99b8d070 req-fa557d57-ce0d-4a1a-8dd0-649d822a3fe7 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:45.738963 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-12e21140-1ac5-4b5b-9ad6-356b99b8d070 req-fa557d57-ce0d-4a1a-8dd0-649d822a3fe7 service nova] Creating event network-changed:bcd94803-b2f3-4f1f-b0ef-275773a5103b for instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e on hv-cinder-maste Sep 18 01:07:45.744801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-12e21140-1ac5-4b5b-9ad6-356b99b8d070 req-fa557d57-ce0d-4a1a-8dd0-649d822a3fe7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098208 Sep 18 01:07:45.745668 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 487/970] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:45.813858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:45.815603 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:45.832484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:45.833538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:45.834509 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:45.837780 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:45.839676 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:45.841618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:45.879692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-3e48cdfd-d87f-4a75-a63d-98638d3f45d6 req-1287474d-e9fd-4113-9ec5-11564f588bb5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c", "tag": "060a9fb2-025c-4f62-be79-f392dbda95a1"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:45.891700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-3e48cdfd-d87f-4a75-a63d-98638d3f45d6 req-1287474d-e9fd-4113-9ec5-11564f588bb5 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:45.892636 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-3e48cdfd-d87f-4a75-a63d-98638d3f45d6 req-1287474d-e9fd-4113-9ec5-11564f588bb5 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:45.893626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-3e48cdfd-d87f-4a75-a63d-98638d3f45d6 req-1287474d-e9fd-4113-9ec5-11564f588bb5 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:45.903949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:07:45.906877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-3e48cdfd-d87f-4a75-a63d-98638d3f45d6 req-1287474d-e9fd-4113-9ec5-11564f588bb5 service nova] Creating event network-changed:060a9fb2-025c-4f62-be79-f392dbda95a1 for instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c on hv-cinder-maste Sep 18 01:07:45.912596 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-3e48cdfd-d87f-4a75-a63d-98638d3f45d6 req-1287474d-e9fd-4113-9ec5-11564f588bb5 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099035 Sep 18 01:07:45.913529 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 488/971] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:07:46.057800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8815e408-8c63-48e7-a3e4-1e411a55cd36 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "DELETE /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 204 len: 0 microversion: 2.1 time: 0.246929 Sep 18 01:07:46.059242 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 484/972] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:07:45 2022] DELETE /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:07:46.070949 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:46.072060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:46.085231 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:46.087243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:46.087953 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:46.088908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:46.089568 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:46.090486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:46.234451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:46.242973 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3bdd5613-ed89-45ce-a7f3-c81958330d56 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1576 microversion: 2.1 time: 0.178253 Sep 18 01:07:46.246505 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 489/973] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:46 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1576 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:47.268537 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:47.269160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:47.275862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:47.276352 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:47.276812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:47.277512 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:47.277966 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:47.278387 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:47.424036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:47.432290 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4cfd4e41-b717-4813-b676-7685c00a64a3 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1576 microversion: 2.1 time: 0.168728 Sep 18 01:07:47.432851 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 485/974] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:47 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1576 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:48.456162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:48.456860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:48.464144 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:48.464748 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:48.465531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:48.466362 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:48.466910 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:48.467404 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:48.600507 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:48.608393 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ba4f5fcd-7008-4ad1-a76e-a664f70194b5 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1457 microversion: 2.1 time: 0.156210 Sep 18 01:07:48.609201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 490/975] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:48 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1457 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:48.674834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-24151925-7ddb-4f95-8d04-eaa2eddae76a req-eb2bb149-d394-49c1-8ff3-10719da0915e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c", "name": "network-vif-deleted", "tag": "060a9fb2-025c-4f62-be79-f392dbda95a1"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:48.686688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-24151925-7ddb-4f95-8d04-eaa2eddae76a req-eb2bb149-d394-49c1-8ff3-10719da0915e service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:48.687169 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-24151925-7ddb-4f95-8d04-eaa2eddae76a req-eb2bb149-d394-49c1-8ff3-10719da0915e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:48.687596 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-24151925-7ddb-4f95-8d04-eaa2eddae76a req-eb2bb149-d394-49c1-8ff3-10719da0915e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:48.699687 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-24151925-7ddb-4f95-8d04-eaa2eddae76a req-eb2bb149-d394-49c1-8ff3-10719da0915e service nova] Creating event network-vif-deleted:060a9fb2-025c-4f62-be79-f392dbda95a1 for instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c on hv-cinder-maste Sep 18 01:07:48.704635 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-24151925-7ddb-4f95-8d04-eaa2eddae76a req-eb2bb149-d394-49c1-8ff3-10719da0915e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033656 Sep 18 01:07:48.705237 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 486/976] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:49.632115 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:49.632737 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:49.639489 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:49.639952 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:49.640390 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:49.641301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:49.641760 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:49.642188 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:49.677854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:49.678347 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:49.678812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:49.679342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lazy-loading 'fault' on Instance uuid 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:07:49.719775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:49.727356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-594482ed-ad13-48b9-9596-ec2154042b3c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 200 len: 1182 microversion: 2.1 time: 0.099263 Sep 18 01:07:49.727919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 491/977] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:49 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 1182 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:50.751543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:50.752160 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:50.758332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:50.758853 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:50.759293 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:50.759978 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:50.760440 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:50.760869 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:50.774291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] HTTP exception thrown: Instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c could not be found. Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Request method failure captured: Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c HTTP/1.1 Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm2QgJpMeVeVcLZ9xjhZKdL8tWRNuqVlXbl2NMu42aYJIy4vR0JFiqcvkHQFss6ZNrK7UGOrxwMS_HU0diziLoPZijNOM_0W3JRqZf6y9fGBmVrMvx3-uc9S57PXVceGxz_bmsOYR3_dBbtQNiKzG6z3tDNmrbRcFwU54fN8ZFm4jaQ Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:07:50.775985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}] Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 823fa0c3eeeb489591c639b81afc02bc Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c could not be found. Sep 18 01:07:50.777655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c'} Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c could not be found. Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:07:50.778540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c could not be found. Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c could not be found. Sep 18 01:07:50.779884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:50.781060 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Returning 404 to user: Instance 6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:07:50.781060 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ba0c2d90-68f3-450a-91ed-648e7e27ca7b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c" status: 404 len: 111 microversion: 2.1 time: 0.030739 Sep 18 01:07:50.781060 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 487/978] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:50 2022] GET /compute/v2.1/servers/6ac6dfe5-30f4-415b-b6aa-7b4f91f95c2c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:07:53.821894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:53.822399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:53.825865 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-99e33997-c751-42e4-b6a3-a53f4ae92631 req-4c70b4e4-582e-4902-ac03-4241c4e077c2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4aaef7df-7526-47f3-a235-e7565d13df51", "tag": "d89c76f8-4e53-4830-b1fe-dff8b48e622b"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:53.829161 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:53.829644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:53.830077 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:53.830850 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:53.831307 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:53.831733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:53.836054 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-99e33997-c751-42e4-b6a3-a53f4ae92631 req-4c70b4e4-582e-4902-ac03-4241c4e077c2 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:53.836625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-99e33997-c751-42e4-b6a3-a53f4ae92631 req-4c70b4e4-582e-4902-ac03-4241c4e077c2 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:53.837136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-99e33997-c751-42e4-b6a3-a53f4ae92631 req-4c70b4e4-582e-4902-ac03-4241c4e077c2 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:53.852962 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-99e33997-c751-42e4-b6a3-a53f4ae92631 req-4c70b4e4-582e-4902-ac03-4241c4e077c2 service nova] Creating event network-changed:d89c76f8-4e53-4830-b1fe-dff8b48e622b for instance 4aaef7df-7526-47f3-a235-e7565d13df51 on hv-cinder-maste Sep 18 01:07:53.858291 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-99e33997-c751-42e4-b6a3-a53f4ae92631 req-4c70b4e4-582e-4902-ac03-4241c4e077c2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036093 Sep 18 01:07:53.859438 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 488/979] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:07:53.859588 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:07:53.914100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:53.945695 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-f13c0295-e565-4df1-b413-785c3c8e99ea tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "DELETE /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 204 len: 0 microversion: 2.1 time: 0.127328 Sep 18 01:07:53.947378 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 492/980] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:07:53 2022] DELETE /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:07:53.957461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:53.958168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:53.968277 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:53.969136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:53.969750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:53.970684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:53.971305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:53.971914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:54.124313 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:54.132279 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4da21739-3bfd-4a70-9e69-ac927ec9025c tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1576 microversion: 2.1 time: 0.179065 Sep 18 01:07:54.132843 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 489/981] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:53 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1576 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:55.156349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:55.157034 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:55.163987 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:55.164464 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:55.164893 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:55.165570 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:55.166017 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:55.166434 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:55.300083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:55.309141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4de6d16f-b57e-42e4-9139-f22c86004275 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1576 microversion: 2.1 time: 0.157049 Sep 18 01:07:55.309905 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 493/982] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:55 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1576 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:56.333358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:56.333963 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:56.341353 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:56.342004 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:56.342658 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:56.343355 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:56.343806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:56.344218 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:56.486484 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:56.488794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:56.496417 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-69bc0fa8-8056-41cf-b60c-33720cddc26b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1457 microversion: 2.1 time: 0.166989 Sep 18 01:07:56.496984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 490/983] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:56 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1457 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:56.709835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-a79989c3-3057-4fb5-a3ef-24537b68b4e2 req-51cb2d6e-fcb5-4b12-ad38-fe731a852bfb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4aaef7df-7526-47f3-a235-e7565d13df51", "name": "network-vif-deleted", "tag": "d89c76f8-4e53-4830-b1fe-dff8b48e622b"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:56.716121 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=62,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:56.722209 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-a79989c3-3057-4fb5-a3ef-24537b68b4e2 req-51cb2d6e-fcb5-4b12-ad38-fe731a852bfb service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:56.722713 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-a79989c3-3057-4fb5-a3ef-24537b68b4e2 req-51cb2d6e-fcb5-4b12-ad38-fe731a852bfb service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:56.723153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-a79989c3-3057-4fb5-a3ef-24537b68b4e2 req-51cb2d6e-fcb5-4b12-ad38-fe731a852bfb service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:56.736074 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-a79989c3-3057-4fb5-a3ef-24537b68b4e2 req-51cb2d6e-fcb5-4b12-ad38-fe731a852bfb service nova] Creating event network-vif-deleted:d89c76f8-4e53-4830-b1fe-dff8b48e622b for instance 4aaef7df-7526-47f3-a235-e7565d13df51 on hv-cinder-maste Sep 18 01:07:56.741935 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-a79989c3-3057-4fb5-a3ef-24537b68b4e2 req-51cb2d6e-fcb5-4b12-ad38-fe731a852bfb service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035807 Sep 18 01:07:56.742621 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 494/984] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:07:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:07:57.520540 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:57.521139 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:57.524531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:07:57.530250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:57.530773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:57.531211 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:57.531914 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:57.532368 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:57.532792 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:57.565331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:57.565814 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:57.566279 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:57.566826 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lazy-loading 'fault' on Instance uuid 4aaef7df-7526-47f3-a235-e7565d13df51 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:07:57.608570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '823fa0c3eeeb489591c639b81afc02bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6958209222b44905bcb4baa1b8c26c58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:57.618482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d05a4d7c-3b51-421c-a959-249510b7235b tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 200 len: 1228 microversion: 2.1 time: 0.101959 Sep 18 01:07:57.619081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 491/985] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:57 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 1228 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:58.643781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:58.644473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 4aaef7df-7526-47f3-a235-e7565d13df51] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:58.651288 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:58.651703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:58.652133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:58.652812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:58.653289 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:58.653680 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:58.667398 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] HTTP exception thrown: Instance 4aaef7df-7526-47f3-a235-e7565d13df51 could not be found. Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Request method failure captured: Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 HTTP/1.1 Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm2QgJpMeVeVcLZ9xjhZKdL8tWRNuqVlXbl2NMu42aYJIy4vR0JFiqcvkHQFss6ZNrK7UGOrxwMS_HU0diziLoPZijNOM_0W3JRqZf6y9fGBmVrMvx3-uc9S57PXVceGxz_bmsOYR3_dBbtQNiKzG6z3tDNmrbRcFwU54fN8ZFm4jaQ Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:07:58.669010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}] Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 823fa0c3eeeb489591c639b81afc02bc Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 4aaef7df-7526-47f3-a235-e7565d13df51 could not be found. Sep 18 01:07:58.670724 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '4aaef7df-7526-47f3-a235-e7565d13df51'} Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4aaef7df-7526-47f3-a235-e7565d13df51 could not be found. Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:07:58.671641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4aaef7df-7526-47f3-a235-e7565d13df51 could not be found. Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4aaef7df-7526-47f3-a235-e7565d13df51 could not be found. Sep 18 01:07:58.673159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:58.674527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Returning 404 to user: Instance 4aaef7df-7526-47f3-a235-e7565d13df51 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:07:58.674527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6ce3a625-7d5a-4a3c-91fe-64499203035f tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51" status: 404 len: 111 microversion: 2.1 time: 0.031842 Sep 18 01:07:58.674527 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 495/986] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:58 2022] GET /compute/v2.1/servers/4aaef7df-7526-47f3-a235-e7565d13df51 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:07:58.821347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:07:58.822011 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:58.828325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:58.828890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:58.829340 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:58.830038 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:58.830528 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:58.831033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:58.854680 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lazy-loading 'flavor' on Instance uuid 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:07:58.927557 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3ec01fe4-3514-4e68-b5ef-9c5bd6279ca5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "POST /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d/action" status: 202 len: 0 microversion: 2.1 time: 0.109885 Sep 18 01:07:58.928135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 492/987] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:07:58 2022] POST /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d/action => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:07:58.938751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:58.939997 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:58.950748 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:58.951524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:58.951975 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:58.952703 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:58.953260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:58.953706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:59.191692 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:07:59.201492 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-5f51c034-a656-4e17-8dad-1818ba0f5931 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.268719 Sep 18 01:07:59.202073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 496/988] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:58 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:07:59.208272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:59.209000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:59.216139 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:59.216618 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:59.217049 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:59.217732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:59.218180 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:59.218642 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:59.231862 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] HTTP exception thrown: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Request method failure captured: Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 HTTP/1.1 Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm2QgJpMeVeVcLZ9xjhZKdL8tWRNuqVlXbl2NMu42aYJIy4vR0JFiqcvkHQFss6ZNrK7UGOrxwMS_HU0diziLoPZijNOM_0W3JRqZf6y9fGBmVrMvx3-uc9S57PXVceGxz_bmsOYR3_dBbtQNiKzG6z3tDNmrbRcFwU54fN8ZFm4jaQ Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:07:59.233508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}] Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 823fa0c3eeeb489591c639b81afc02bc Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.235214 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '443c0c33-d16f-4b9b-86dc-99207ed72962'} Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:07:59.236184 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.237631 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.239178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Returning 404 to user: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:07:59.239178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-572678b4-5d0a-4371-9fb4-20c8f078e6ce tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "DELETE /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 404 len: 111 microversion: 2.1 time: 0.031059 Sep 18 01:07:59.239178 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 493/989] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:07:59 2022] DELETE /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:07:59.244134 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:07:59.244739 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] [instance: 443c0c33-d16f-4b9b-86dc-99207ed72962] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:07:59.250788 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:59.251274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:59.251706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:59.252388 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:07:59.252862 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:07:59.253282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:07:59.266793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] HTTP exception thrown: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Request method failure captured: Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 HTTP/1.1 Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm2QgJpMeVeVcLZ9xjhZKdL8tWRNuqVlXbl2NMu42aYJIy4vR0JFiqcvkHQFss6ZNrK7UGOrxwMS_HU0diziLoPZijNOM_0W3JRqZf6y9fGBmVrMvx3-uc9S57PXVceGxz_bmsOYR3_dBbtQNiKzG6z3tDNmrbRcFwU54fN8ZFm4jaQ Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:07:59.268304 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6958209222b44905bcb4baa1b8c26c58"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6958209222b44905bcb4baa1b8c26c58"}]}] Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 6958209222b44905bcb4baa1b8c26c58 Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-TestVolumeBootPattern-1926783189 Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 823fa0c3eeeb489591c639b81afc02bc Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-TestVolumeBootPattern-1926783189-project Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.269926 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '443c0c33-d16f-4b9b-86dc-99207ed72962'} Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:07:59.270922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. Sep 18 01:07:59.272504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:07:59.274766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Returning 404 to user: Instance 443c0c33-d16f-4b9b-86dc-99207ed72962 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:07:59.274766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-006f88a5-0edb-4f2d-a9c3-c71833f86926 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962" status: 404 len: 111 microversion: 2.1 time: 0.029889 Sep 18 01:07:59.274766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 497/990] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:07:59 2022] GET /compute/v2.1/servers/443c0c33-d16f-4b9b-86dc-99207ed72962 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:08:00.225776 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:00.226936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:00.233309 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:00.233794 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:00.234226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:00.234951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:00.235408 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:00.235832 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:00.369122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:00.376903 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-0a9a4186-af43-4432-9b6b-bc0360294813 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.155119 Sep 18 01:08:00.377461 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 494/991] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:00 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:00.562153 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-fe2fa506-273e-401e-b4a2-dc7dad909b30 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] Calling method '.version_select at 0x7fcab1ac54c0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:00.581601 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-fe2fa506-273e-401e-b4a2-dc7dad909b30 tempest-TestVolumeBootPattern-1926783189 tempest-TestVolumeBootPattern-1926783189-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-58756093" status: 202 len: 0 microversion: 2.1 time: 0.023208 Sep 18 01:08:00.582176 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 498/992] 10.222.0.42 () {58 vars in 1257 bytes} [Sun Sep 18 01:08:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-58756093 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 18 01:08:01.400768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:01.401329 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:01.407665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:01.408140 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:01.408568 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:01.409267 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:01.409704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:01.410122 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:01.621101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:01.628944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d8a83bbd-776c-4997-9acf-a917e3d31732 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.232299 Sep 18 01:08:01.629521 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 495/993] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:01 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:02.652564 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:02.653325 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:02.660895 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:02.661391 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:02.661842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:02.662564 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:02.663024 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:02.663453 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:02.882285 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:02.890522 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ca94b77a-d0af-43ca-bb74-4d4b49e8270a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.242547 Sep 18 01:08:02.891349 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 499/994] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:02 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:08:03.913250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:03.913901 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:03.919887 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:03.920371 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:03.920806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:03.921502 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:03.921955 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:03.922380 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:04.072980 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:04.080806 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-405f206a-9be0-4314-91e4-1a0e3ad40fdb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.171601 Sep 18 01:08:04.081391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 496/995] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:03 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:05.105343 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:05.106004 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:05.112508 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:05.113068 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:05.113576 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:05.114338 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:05.114885 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:05.115401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:05.252727 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:05.260816 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a7a2d62b-9405-4775-9e45-5c796351def4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.159934 Sep 18 01:08:05.261541 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 500/996] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:05 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:06.287254 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:06.287900 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:06.294855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:06.295399 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:06.295834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:06.296531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:06.296986 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:06.297465 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:06.445306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:06.453760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-098ce391-ea2d-4be0-8edb-48e7cef536e4 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.170295 Sep 18 01:08:06.454369 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 497/997] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:06 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:07.477463 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:07.478342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:07.484750 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:07.485236 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:07.485665 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:07.486368 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:07.486863 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:07.487315 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:07.648291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:07.656259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-242fc251-a619-4bad-9a63-05be0ede732b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.182577 Sep 18 01:08:07.656841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 501/998] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:07 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:08.679820 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:08.680453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:08.687022 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:08.687507 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:08.687948 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:08.688636 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:08.689088 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:08.689510 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:08.823456 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:08.831081 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7ea4769a-eefa-4d9f-a570-a78573ad75b8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.155219 Sep 18 01:08:08.831648 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 498/999] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:08 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:09.855847 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:09.856387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:09.862805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:09.863291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:09.863730 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:09.864444 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:09.864882 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:09.865307 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:10.000233 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:10.008012 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3dcb588e-f90f-4569-93de-52a27834aed8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.156794 Sep 18 01:08:10.008567 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 502/1000] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:09 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:11.031603 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:11.032212 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:11.039465 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:11.039951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:11.040396 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:11.041092 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:11.041545 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:11.041965 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:11.177443 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:11.185860 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-27b94077-be07-4142-8f8c-8d47a59bd8f0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.158136 Sep 18 01:08:11.186418 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 499/1001] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:11 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:12.208957 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:12.209562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:12.216310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:12.216798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:12.217229 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:12.217915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:12.218399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:12.218849 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:12.352826 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:12.360872 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8c297253-7b4a-4aa0-8085-5f22162c68cb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1569 microversion: 2.1 time: 0.155752 Sep 18 01:08:12.361451 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 503/1002] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:12 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:12.384255 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:08:12.384904 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:12.391941 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:12.392445 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:12.392871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:12.393585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:12.394044 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:12.394492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:12.439572 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120832) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 18 01:08:12.506770 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-90174f41-8f22-4f6a-b382-0d095bdfe608 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "POST /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d/action" status: 202 len: 0 microversion: 2.1 time: 0.126211 Sep 18 01:08:12.507382 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 500/1003] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 18 01:08:12 2022] POST /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d/action => generated 0 bytes in 128 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:08:12.515674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:12.516486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:12.525053 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:12.525551 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:12.526197 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:12.526932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:12.527387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:12.527838 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:12.660638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:12.669210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-aa6d2db1-df24-4e9a-ae78-58025cc4af47 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1517 microversion: 2.1 time: 0.157328 Sep 18 01:08:12.669779 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 504/1004] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:12 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1517 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:13.692632 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:13.693238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:13.700391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:13.700878 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:13.701369 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:13.702102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:13.702584 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:13.703009 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:13.836095 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:13.844096 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9bd4658f-cf37-4126-a72c-79e5c82e949a tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1517 microversion: 2.1 time: 0.155401 Sep 18 01:08:13.844673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 501/1005] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:13 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1517 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:14.868394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:14.868993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:14.875441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:14.875920 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:14.876357 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:14.877031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:14.877504 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:14.877924 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:15.010591 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:15.018280 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2edb0361-3b91-4dd7-9108-a2ed0c53d667 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1517 microversion: 2.1 time: 0.153732 Sep 18 01:08:15.018871 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 505/1006] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:14 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1517 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:16.046704 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:16.047352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:16.054717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:16.055354 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:16.055883 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:16.057542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:16.057542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:16.057542 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:16.117460 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-b4382376-c8b1-44ff-a86d-411f65bfbce1 req-883f9590-5c7c-48a9-b8ab-6d44f1bad897 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69223fa4-55c0-4c65-b9d4-c2e5ec0b778d", "name": "network-vif-unplugged", "status": "completed", "tag": "ac27aed2-9401-48f5-a764-f83ca47284b4"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:08:16.126974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b4382376-c8b1-44ff-a86d-411f65bfbce1 req-883f9590-5c7c-48a9-b8ab-6d44f1bad897 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:16.127537 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b4382376-c8b1-44ff-a86d-411f65bfbce1 req-883f9590-5c7c-48a9-b8ab-6d44f1bad897 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:16.128041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-b4382376-c8b1-44ff-a86d-411f65bfbce1 req-883f9590-5c7c-48a9-b8ab-6d44f1bad897 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:16.142332 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.compute.server_external_events [req-b4382376-c8b1-44ff-a86d-411f65bfbce1 req-883f9590-5c7c-48a9-b8ab-6d44f1bad897 service nova] Unable to find a host for instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d. Dropping event network-vif-unplugged {{(pid=120832) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 18 01:08:16.143238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-b4382376-c8b1-44ff-a86d-411f65bfbce1 req-883f9590-5c7c-48a9-b8ab-6d44f1bad897 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.107061 Sep 18 01:08:16.143837 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 502/1007] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:08:16 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 108 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 18 01:08:16.194894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:16.203139 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-91e2d5df-3ba2-4b13-a183-4cd17ca5316f tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1517 microversion: 2.1 time: 0.160372 Sep 18 01:08:16.203702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 506/1008] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:16 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1517 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:17.227775 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:17.228422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:17.234883 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:17.235363 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:17.235803 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:17.236485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:17.236935 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:17.237356 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:17.367386 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:17.375259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8cd7cb72-26cd-4ced-bfe6-cb4ee1a06589 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1517 microversion: 2.1 time: 0.151922 Sep 18 01:08:17.375818 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 503/1009] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:17 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1517 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:18.397836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:18.398595 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:18.405106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:18.405595 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:18.406032 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:18.407270 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:18.407270 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:18.407621 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:18.547945 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:18.556274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2536b64c-8c7d-4b67-9ad4-fef59d1d7bac tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1519 microversion: 2.1 time: 0.162342 Sep 18 01:08:18.556836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 507/1010] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:18 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1519 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:19.579585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:19.580191 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:19.587056 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:19.587541 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:19.587970 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:19.588657 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:19.589104 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:19.589522 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:19.720479 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:19.728247 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4ad0a842-beff-4a14-a76b-c1442324e07b tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.152598 Sep 18 01:08:19.728924 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 504/1011] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:19 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:08:20.751979 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:20.752517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:20.759480 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:20.759956 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:20.760385 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:20.761067 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:20.761518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:20.762001 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:20.890145 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:20.897834 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-48b1a3da-4c78-429e-8d1c-cd1c8ec611d9 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.149860 Sep 18 01:08:20.898394 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 508/1012] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:20 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:21.884017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:21.884587 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:21.891240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:21.891715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:21.892139 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:21.892819 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:21.893268 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:21.893685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:21.908269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:08:21.924151 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1ba4736f-4bf4-4e4a-a70f-7189bdb54202 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/ips" status: 200 len: 145 microversion: 2.1 time: 0.043868 Sep 18 01:08:21.924746 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 505/1013] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 18 01:08:21 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/ips => generated 145 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:21.926654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:21.927285 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:21.934149 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:21.934666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:21.935096 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:21.935788 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:21.936237 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:21.936753 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:22.071929 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:22.080474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c411fab6-d58f-4988-b5e1-bfd2a96ccace tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.157968 Sep 18 01:08:22.081245 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 509/1014] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:21 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:08:23.105590 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:23.106232 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:23.115051 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:23.115538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:23.115974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:23.116670 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:23.117131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:23.118305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:23.251776 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:23.259805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-440683dd-78f1-472a-bc40-675c1426750c tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.158535 Sep 18 01:08:23.260374 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 506/1015] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:23 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:23.261934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0a417e06-012c-48cd-90a1-c472a0401113"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:08:23.264328 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:23.270798 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:23.271282 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:23.271712 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:23.272404 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:23.272854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:23.273279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:23.551150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-41d61812-7f32-46fc-a637-692d9feedf04 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.293071 Sep 18 01:08:23.551753 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 510/1016] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:08:23 2022] POST /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments => generated 194 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:08:24.283347 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:24.284139 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:24.291335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:24.291828 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:24.292250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:24.293241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:24.293392 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:24.293812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:24.427287 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:24.435727 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cc7b9999-74d3-4f6f-a835-afbb9e2af2b5 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.156271 Sep 18 01:08:24.436326 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 507/1017] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:24 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:25.460284 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:25.460975 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:25.467494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:25.467969 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:25.468401 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:25.469082 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:25.469556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:25.469941 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:25.472645 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:08:25.608189 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:25.615900 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9da5fa64-9cb1-44fd-8325-5a90615c13d8 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.160330 Sep 18 01:08:25.616471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 511/1018] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:25 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:26.639222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:26.639835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:26.646296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:26.646804 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:26.647242 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:26.647928 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:26.648376 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:26.648800 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:26.777473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:26.785522 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a87516e4-8a02-4d3e-bf13-68bb121f41fb tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.150204 Sep 18 01:08:26.786074 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 508/1019] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:26 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:27.808944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:27.809553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:27.816109 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:27.816585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:27.817020 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:27.817709 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:27.818165 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:27.818622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:27.952466 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:27.960334 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d4ce1b5b-f212-480a-9906-3f424e1ad3ae tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.155800 Sep 18 01:08:27.960914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 512/1020] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:27 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:28.983750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:28.984274 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:28.986697 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:08:28.991951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:28.992436 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:28.992871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:28.993645 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:28.994123 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:28.994567 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:29.131645 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:29.140174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c3ad03cf-6ba0-4ec3-a04e-4ddd46a4c3c0 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.160318 Sep 18 01:08:29.140738 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 509/1021] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:28 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:30.126337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-942faa22-ed86-4425-b200-5c82f887ebe9 req-86317d55-8d2b-47eb-bca3-1d1e36e9eba0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69223fa4-55c0-4c65-b9d4-c2e5ec0b778d", "name": "network-vif-plugged", "status": "completed", "tag": "ac27aed2-9401-48f5-a764-f83ca47284b4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:08:30.132095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=39,nova_api:DELETE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:08:30.138056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-942faa22-ed86-4425-b200-5c82f887ebe9 req-86317d55-8d2b-47eb-bca3-1d1e36e9eba0 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:30.138594 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-942faa22-ed86-4425-b200-5c82f887ebe9 req-86317d55-8d2b-47eb-bca3-1d1e36e9eba0 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:30.139274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-942faa22-ed86-4425-b200-5c82f887ebe9 req-86317d55-8d2b-47eb-bca3-1d1e36e9eba0 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:30.158938 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-942faa22-ed86-4425-b200-5c82f887ebe9 req-86317d55-8d2b-47eb-bca3-1d1e36e9eba0 service nova] Creating event network-vif-plugged:ac27aed2-9401-48f5-a764-f83ca47284b4 for instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d on hv-cinder-maste Sep 18 01:08:30.166503 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-942faa22-ed86-4425-b200-5c82f887ebe9 req-86317d55-8d2b-47eb-bca3-1d1e36e9eba0 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042809 Sep 18 01:08:30.167228 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:30.167846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:30.169178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 513/1022] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:08:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:30.175084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:30.175726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:30.176179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:30.176884 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:30.177342 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:30.177768 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:30.330478 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:30.339611 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e5f92fc7-382b-4970-bfa9-35bf71489862 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.178597 Sep 18 01:08:30.340086 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 510/1023] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:30 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:31.097812 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:31.100075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:31.106785 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:31.107269 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:31.107706 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:31.108393 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:31.108846 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:31.109267 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:31.141355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ef6823d8-9352-4f51-830e-0a6c96b494bb tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048644 Sep 18 01:08:31.142008 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 514/1024] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:08:31 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:08:31.152746 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:31.153233 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:31.159182 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:31.159665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:31.160100 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:31.160792 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:31.161238 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:31.161659 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:31.191259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b6103133-e16b-4ef0-b6d8-502c9140852e tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/0a417e06-012c-48cd-90a1-c472a0401113" status: 200 len: 194 microversion: 2.1 time: 0.042524 Sep 18 01:08:31.191901 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 511/1025] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Sep 18 01:08:31 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/0a417e06-012c-48cd-90a1-c472a0401113 => generated 194 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:31.363568 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:31.364299 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:31.371946 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:31.372524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:31.372973 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:31.373676 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:31.374134 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:31.374584 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:31.520974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:31.530023 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d9508545-2edd-4f60-beb3-f95fef844528 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1575 microversion: 2.1 time: 0.170615 Sep 18 01:08:31.530658 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 515/1026] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:31 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1575 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:32.522654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cdb2eefa-b474-4f92-aa3d-7eec5581d67f"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:08:32.525813 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:32.532433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:32.532922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:32.533359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:32.534062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:32.534542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:32.534974 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:32.554717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:32.555479 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:32.561634 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:32.562112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:32.562572 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:32.563259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:32.563707 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:32.564166 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:32.703461 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:32.711784 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1626eced-7b3f-4d72-9117-ad479e292d88 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1562 microversion: 2.1 time: 0.161981 Sep 18 01:08:32.712993 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 516/1027] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:32 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1562 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:08:32.796701 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-d8538b6f-d168-431e-94ef-a6fc45d409d5 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "POST /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.277957 Sep 18 01:08:32.797394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 512/1028] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:08:32 2022] POST /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments => generated 194 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:40.331303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:40.333471 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:40.340809 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:40.341292 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:40.341718 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:40.342702 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:40.343152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:40.343569 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:40.372888 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-17e6e44b-5270-42b0-a185-e148faf77432 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.045615 Sep 18 01:08:40.373556 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 517/1029] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:08:40 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments => generated 371 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:40.517246 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:40.517833 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:40.524272 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:40.524763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:40.525203 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:40.525913 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:40.526370 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:40.526945 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:40.558685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1e422b59-af90-4289-ad94-7bdbadf728b9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/0a417e06-012c-48cd-90a1-c472a0401113" status: 200 len: 194 microversion: 2.1 time: 0.046402 Sep 18 01:08:40.559273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 513/1030] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Sep 18 01:08:40 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/0a417e06-012c-48cd-90a1-c472a0401113 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:40.569622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:40.570113 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:40.576714 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:40.577197 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:40.577750 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:40.578318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:40.578795 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:40.579221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:40.659079 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lazy-loading 'services' on Instance uuid fc403fe0-bf54-4ff8-b714-0cc3f0484d1e {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:08:40.730474 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-75b488a6-135d-4dee-88b9-73daefc3f705 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/0a417e06-012c-48cd-90a1-c472a0401113" status: 202 len: 0 microversion: 2.1 time: 0.164517 Sep 18 01:08:40.731078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 518/1031] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:08:40 2022] DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/0a417e06-012c-48cd-90a1-c472a0401113 => generated 0 bytes in 166 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:08:42.909452 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:42.909982 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:42.917057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:42.917534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:42.917960 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:42.918677 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:42.919124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:42.919545 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:42.947358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-a3a541c5-d76a-46a4-a026-f3eca70cdfaa tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/cdb2eefa-b474-4f92-aa3d-7eec5581d67f" status: 200 len: 194 microversion: 2.1 time: 0.041802 Sep 18 01:08:42.947857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 514/1032] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Sep 18 01:08:42 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/cdb2eefa-b474-4f92-aa3d-7eec5581d67f => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:42.959428 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:42.960010 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:42.966686 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:42.967439 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:42.967909 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:42.968592 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:42.969045 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:42.969467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:43.048726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lazy-loading 'services' on Instance uuid fc403fe0-bf54-4ff8-b714-0cc3f0484d1e {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:08:43.120918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c0699889-2aa6-4af9-90e0-5e3bc2c0ce89 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/cdb2eefa-b474-4f92-aa3d-7eec5581d67f" status: 202 len: 0 microversion: 2.1 time: 0.166057 Sep 18 01:08:43.121505 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 519/1033] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:08:42 2022] DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments/cdb2eefa-b474-4f92-aa3d-7eec5581d67f => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:08:44.300492 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:44.302753 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:44.309037 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.309512 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.309948 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.310665 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.311113 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.311538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.340226 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-cdaff445-1d5e-41c9-a456-5a062c232a64 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043597 Sep 18 01:08:44.340778 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 515/1034] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:08:44 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:08:44.455899 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:44.458295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:44.465711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.466203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.466678 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.467641 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.468099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.468518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.497889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-40c7f23b-0569-42a2-8f5e-3370efd1550c tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046221 Sep 18 01:08:44.498370 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 520/1035] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 18 01:08:44 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 18 01:08:44.561893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:44.562388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:44.568294 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.568775 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.569212 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.569900 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.570355 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.570801 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.596730 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:08:44.672443 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2a371e77-6ce9-44ec-b8eb-40a0cfcf2a18 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 204 len: 0 microversion: 2.1 time: 0.114017 Sep 18 01:08:44.673139 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 516/1036] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:08:44 2022] DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:08:44.684138 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:44.684751 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:44.690717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.691207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.691643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.692344 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:44.692786 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:44.693207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:44.824919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:44.832554 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a592f881-e8bb-4d57-8037-8579c5672c91 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1673 microversion: 2.1 time: 0.154235 Sep 18 01:08:44.833112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 521/1037] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:44 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1673 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:45.857010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:45.857707 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:45.864104 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:45.864583 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:45.865019 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:45.865709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:45.866162 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:45.866630 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:45.991958 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:45.999586 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1fc8fa57-65d5-4fbd-991f-02d28f631008 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1673 microversion: 2.1 time: 0.147569 Sep 18 01:08:46.000146 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 517/1038] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:45 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1673 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:47.023207 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:47.023799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:47.030761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:47.031247 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:47.031678 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:47.032371 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:47.032881 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:47.033310 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:47.165960 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:47.173925 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e256efe6-081f-49d7-93bf-9c8f048f1eab tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1673 microversion: 2.1 time: 0.154570 Sep 18 01:08:47.174510 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 522/1039] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:47 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1673 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:47.625916 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-ea6d8fda-1e46-41b0-81da-f7d9136482b1 req-39a30327-9c74-4db0-8b71-b28c5c5fb806 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc403fe0-bf54-4ff8-b714-0cc3f0484d1e", "name": "network-vif-deleted", "tag": "bcd94803-b2f3-4f1f-b0ef-275773a5103b"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:08:47.636719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-ea6d8fda-1e46-41b0-81da-f7d9136482b1 req-39a30327-9c74-4db0-8b71-b28c5c5fb806 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:47.637196 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-ea6d8fda-1e46-41b0-81da-f7d9136482b1 req-39a30327-9c74-4db0-8b71-b28c5c5fb806 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:47.637628 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-ea6d8fda-1e46-41b0-81da-f7d9136482b1 req-39a30327-9c74-4db0-8b71-b28c5c5fb806 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:47.651023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-ea6d8fda-1e46-41b0-81da-f7d9136482b1 req-39a30327-9c74-4db0-8b71-b28c5c5fb806 service nova] Creating event network-vif-deleted:bcd94803-b2f3-4f1f-b0ef-275773a5103b for instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e on hv-cinder-maste Sep 18 01:08:47.656092 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-ea6d8fda-1e46-41b0-81da-f7d9136482b1 req-39a30327-9c74-4db0-8b71-b28c5c5fb806 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034220 Sep 18 01:08:47.656787 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 518/1040] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:08:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:08:48.197922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:48.198544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:48.204932 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:48.205735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:48.206170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:48.207203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:48.207828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:48.208257 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:48.240594 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:48.241079 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:48.241506 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:48.241972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lazy-loading 'fault' on Instance uuid fc403fe0-bf54-4ff8-b714-0cc3f0484d1e {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:08:48.286597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd2fa2deb0a24aff9d8b7ce97b6e461b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '39509a7e0ca64f94913b5fbf10c6ec9c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:08:48.294990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-eba45816-6e23-4dfa-989b-b76cf4efb95d tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 200 len: 1327 microversion: 2.1 time: 0.101500 Sep 18 01:08:48.298514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 523/1041] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:48 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 1327 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:08:49.320222 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:49.320920 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:49.327124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:49.327602 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:49.328036 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:49.328720 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:49.329173 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:49.329591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:49.342898 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] HTTP exception thrown: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Request method failure captured: Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e HTTP/1.1 Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm6hL0Uvh_1oiIthReAaa8UVyGZ00HAmRKJVSA9KcTBMYrkxbCudVVKZt_k1amCBU1w7HbwCfq9E2yzKMEGCAxyV3xbUCcyh-bovIhA-5gNbBMCKf6fK9mnUu3NDOx0crgBOqdQLV9VT8x1QxI6U_u4rXOadhkMkBmFBSg4o9MkYGBo Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:08:49.344494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}] Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: bd2fa2deb0a24aff9d8b7ce97b6e461b Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:49.346136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': 'fc403fe0-bf54-4ff8-b714-0cc3f0484d1e'} Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:08:49.347133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:49.348689 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:49.350101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Returning 404 to user: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:08:49.350101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c605a9be-0353-4165-b6e6-1b815695763f tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 404 len: 111 microversion: 2.1 time: 0.030299 Sep 18 01:08:49.350101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 519/1042] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:49 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:08:49.355801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-2a84f64d-9320-4ae4-9cc5-8add36f0ee8b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '.version_select at 0x7fcab1a25ca0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:49.375959 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-2a84f64d-9320-4ae4-9cc5-8add36f0ee8b tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-356981727" status: 202 len: 0 microversion: 2.1 time: 0.024405 Sep 18 01:08:49.376518 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 524/1043] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 18 01:08:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-356981727 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:08:52.252306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:52.252846 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:52.259389 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:52.259873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:52.260306 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:52.260997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:52.261455 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:52.261876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:52.275175 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] HTTP exception thrown: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Request method failure captured: Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e HTTP/1.1 Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm6hL0Uvh_1oiIthReAaa8UVyGZ00HAmRKJVSA9KcTBMYrkxbCudVVKZt_k1amCBU1w7HbwCfq9E2yzKMEGCAxyV3xbUCcyh-bovIhA-5gNbBMCKf6fK9mnUu3NDOx0crgBOqdQLV9VT8x1QxI6U_u4rXOadhkMkBmFBSg4o9MkYGBo Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:08:52.276747 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}] Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: bd2fa2deb0a24aff9d8b7ce97b6e461b Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.278422 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': 'fc403fe0-bf54-4ff8-b714-0cc3f0484d1e'} Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:08:52.279312 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.280585 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.282220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Returning 404 to user: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:08:52.282220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4c080e8c-a23b-4e27-926b-305dabb6add9 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 404 len: 111 microversion: 2.1 time: 0.029731 Sep 18 01:08:52.282220 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 520/1044] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:08:52 2022] DELETE /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:08:52.286684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:52.287290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: fc403fe0-bf54-4ff8-b714-0cc3f0484d1e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:52.295060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:52.295553 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:52.295974 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:52.296657 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:52.297106 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:52.297524 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:52.310943 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] HTTP exception thrown: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Request method failure captured: Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e HTTP/1.1 Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm6hL0Uvh_1oiIthReAaa8UVyGZ00HAmRKJVSA9KcTBMYrkxbCudVVKZt_k1amCBU1w7HbwCfq9E2yzKMEGCAxyV3xbUCcyh-bovIhA-5gNbBMCKf6fK9mnUu3NDOx0crgBOqdQLV9VT8x1QxI6U_u4rXOadhkMkBmFBSg4o9MkYGBo Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:08:52.312573 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}] Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: bd2fa2deb0a24aff9d8b7ce97b6e461b Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.314167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'fc403fe0-bf54-4ff8-b714-0cc3f0484d1e'} Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:08:52.315132 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. Sep 18 01:08:52.316638 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:52.317984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Returning 404 to user: Instance fc403fe0-bf54-4ff8-b714-0cc3f0484d1e could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:08:52.317984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-77cf8a05-c6a8-48b0-b8e8-b54b1e0c57ef tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e" status: 404 len: 111 microversion: 2.1 time: 0.031130 Sep 18 01:08:52.317984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 525/1045] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:52 2022] GET /compute/v2.1/servers/fc403fe0-bf54-4ff8-b714-0cc3f0484d1e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:08:53.455760 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:53.456239 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:53.462782 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:53.463269 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:53.463739 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:53.464435 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:53.464890 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:53.465316 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:53.468324 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:08:53.481077 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] HTTP exception thrown: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Request method failure captured: Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: DELETE /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 HTTP/1.1 Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm6hL0Uvh_1oiIthReAaa8UVyGZ00HAmRKJVSA9KcTBMYrkxbCudVVKZt_k1amCBU1w7HbwCfq9E2yzKMEGCAxyV3xbUCcyh-bovIhA-5gNbBMCKf6fK9mnUu3NDOx0crgBOqdQLV9VT8x1QxI6U_u4rXOadhkMkBmFBSg4o9MkYGBo Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:08:53.482835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}] Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: bd2fa2deb0a24aff9d8b7ce97b6e461b Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.484444 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '529a4d99-a333-4068-a419-09d67a1ae170'} Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:08:53.485346 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.486655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.487956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Returning 404 to user: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:08:53.487956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e52557e5-d79c-410d-9865-ef45cef7b8b4 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "DELETE /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 404 len: 111 microversion: 2.1 time: 0.032406 Sep 18 01:08:53.487956 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 521/1046] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:08:53 2022] DELETE /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:08:53.492983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:53.494136 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] [instance: 529a4d99-a333-4068-a419-09d67a1ae170] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:08:53.500544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:53.501031 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:53.501472 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:53.502162 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:08:53.502660 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:08:53.503090 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:08:53.516990 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] HTTP exception thrown: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Request method failure captured: Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 HTTP/1.1 Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm6hL0Uvh_1oiIthReAaa8UVyGZ00HAmRKJVSA9KcTBMYrkxbCudVVKZt_k1amCBU1w7HbwCfq9E2yzKMEGCAxyV3xbUCcyh-bovIhA-5gNbBMCKf6fK9mnUu3NDOx0crgBOqdQLV9VT8x1QxI6U_u4rXOadhkMkBmFBSg4o9MkYGBo Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:08:53.518544 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_39509a7e0ca64f94913b5fbf10c6ec9c"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/39509a7e0ca64f94913b5fbf10c6ec9c"}]}] Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: 39509a7e0ca64f94913b5fbf10c6ec9c Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-AttachVolumeTestJSON-243269240 Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: bd2fa2deb0a24aff9d8b7ce97b6e461b Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-AttachVolumeTestJSON-243269240-project Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.519972 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '529a4d99-a333-4068-a419-09d67a1ae170'} Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:08:53.520789 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. Sep 18 01:08:53.522013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:08:53.523211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] Returning 404 to user: Instance 529a4d99-a333-4068-a419-09d67a1ae170 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:08:53.523211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1c83a4f8-8414-4054-a5d5-9adb0b8a1b28 tempest-AttachVolumeTestJSON-243269240 tempest-AttachVolumeTestJSON-243269240-project] 10.222.0.42 "GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170" status: 404 len: 111 microversion: 2.1 time: 0.031013 Sep 18 01:08:53.523211 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 526/1047] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:08:53 2022] GET /compute/v2.1/servers/529a4d99-a333-4068-a419-09d67a1ae170 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:08:58.888398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-32aa75fc-ea89-4714-bdbb-46160b762593 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1c468b0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:08:58.934979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-32aa75fc-ea89-4714-bdbb-46160b762593 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/781d8ca4-ba46-4000-8215-a4df5bd0d93f" status: 200 len: 351 microversion: 2.1 time: 0.050588 Sep 18 01:08:58.935538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 522/1048] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:08:58 2022] GET /compute/v2.1/os-volumes/781d8ca4-ba46-4000-8215-a4df5bd0d93f => generated 351 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:00.268807 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-46b17222-a0c9-4f1b-a870-411fa02bd156 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1d624c0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:00.318264 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-46b17222-a0c9-4f1b-a870-411fa02bd156 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/f1d0c2ce-6f76-4a34-84e3-f1508d8a3321" status: 200 len: 351 microversion: 2.1 time: 0.053253 Sep 18 01:09:00.318873 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 527/1049] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:09:00 2022] GET /compute/v2.1/os-volumes/f1d0c2ce-6f76-4a34-84e3-f1508d8a3321 => generated 351 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:01.649719 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-2d7e3a7d-9491-4b89-8a50-c26ea38509a4 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1c98820>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:01.695099 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-2d7e3a7d-9491-4b89-8a50-c26ea38509a4 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/276351dd-5e90-41d8-b1ff-84fd850abc44" status: 200 len: 350 microversion: 2.1 time: 0.049603 Sep 18 01:09:01.695656 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 523/1050] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 18 01:09:01 2022] GET /compute/v2.1/os-volumes/276351dd-5e90-41d8-b1ff-84fd850abc44 => generated 350 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:01.708913 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a12bbfcf-17b3-4f85-92d6-b85298b68596 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1b69280>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:01.792175 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a12bbfcf-17b3-4f85-92d6-b85298b68596 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.087468 Sep 18 01:09:01.792830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 528/1051] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Sep 18 01:09:01 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:01.806626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b7f933a0-ffad-4918-bcfd-d96b388b56be tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1c98ee0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:01.888407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b7f933a0-ffad-4918-bcfd-d96b388b56be tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.085546 Sep 18 01:09:01.888997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 524/1052] 10.222.0.42 () {58 vars in 1149 bytes} [Sun Sep 18 01:09:01 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:01.903252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9f36ce79-3f3b-41d0-8618-37605c50ea20 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1b691f0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:01.984352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9f36ce79-3f3b-41d0-8618-37605c50ea20 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.085484 Sep 18 01:09:01.984914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 529/1053] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Sep 18 01:09:01 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:01.997985 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-1dd1fc2f-36f1-4f33-930c-c2df4f3632c4 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1aed310>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:02.086042 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-1dd1fc2f-36f1-4f33-930c-c2df4f3632c4 tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.091709 Sep 18 01:09:02.086641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 525/1054] 10.222.0.42 () {58 vars in 1167 bytes} [Sun Sep 18 01:09:01 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:02.101335 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6ad468ee-8b5a-4912-a5da-7ba80d7e5fcc tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1b44040>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:02.182922 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6ad468ee-8b5a-4912-a5da-7ba80d7e5fcc tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.084774 Sep 18 01:09:02.183491 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 530/1055] 10.222.0.42 () {58 vars in 1170 bytes} [Sun Sep 18 01:09:02 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:02.197023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c5a09d93-7131-48d9-96e0-76f9dafd2bac tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1bdeca0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:02.277818 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c5a09d93-7131-48d9-96e0-76f9dafd2bac tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.084469 Sep 18 01:09:02.278405 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 526/1056] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Sep 18 01:09:02 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:02.291715 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-9570c18c-6e78-4ba5-b673-da52217dc53b tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1b870d0>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:02.375721 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-9570c18c-6e78-4ba5-b673-da52217dc53b tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.087771 Sep 18 01:09:02.376293 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 531/1057] 10.222.0.42 () {58 vars in 1188 bytes} [Sun Sep 18 01:09:02 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:02.390194 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ee7ff46a-2c45-4346-91b3-ef62e577473c tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] Calling method '.version_select at 0x7fcab1c988b0>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:02.472389 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ee7ff46a-2c45-4346-91b3-ef62e577473c tempest-VolumesTestJSON-2100581533 tempest-VolumesTestJSON-2100581533-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.085903 Sep 18 01:09:02.472953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 527/1058] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Sep 18 01:09:02 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:25.603861 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=30,nova_api:DELETE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:25.604605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=85,nova_cell1:INSERT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:25.609924 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1933732539", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "networks": [{"uuid": "d37bfa6d-c1b4-4c46-b4de-962869a5a79e"}]}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:25.665294 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.network.neutron [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] validate_networks() for [('d37bfa6d-c1b4-4c46-b4de-962869a5a79e', None, None, None, None, None)] {{(pid=120831) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:09:26.041980 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Flavor limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:09:26.042409 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Image limits 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:09:26.042914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Flavor pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:09:26.043318 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Image pref 0:0:0 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:09:26.043733 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.virt.hardware [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120831) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:09:26.052436 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:26.052919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:26.053350 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:26.054079 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:26.054562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:26.054988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:26.072543 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Getting quotas for project c83778305ed1494f931ca66d197fa5d3. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:09:26.076674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.quota [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Getting quotas for user 33ea64ca8ec14dd2a34a05ce90f9fa7a and project c83778305ed1494f931ca66d197fa5d3. Resources: {'cores', 'ram', 'instances'} {{(pid=120831) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:09:26.088309 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Going to run 1 instances... {{(pid=120831) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:09:26.090644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120831) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:09:26.136492 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e468ffe4-ab91-4fc0-83bc-ed57b4ae63f8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.533868 Sep 18 01:09:26.137150 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 532/1059] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:09:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 535 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:09:26.146483 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:26.147508 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:26.152138 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-9781d1aa-fbd3-44ce-923c-5f400c27dd02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:26.152759 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-9781d1aa-fbd3-44ce-923c-5f400c27dd02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:26.211767 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-9781d1aa-fbd3-44ce-923c-5f400c27dd02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:26.218868 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-9781d1aa-fbd3-44ce-923c-5f400c27dd02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1214 microversion: 2.1 time: 0.073336 Sep 18 01:09:26.219439 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 528/1060] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:26 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1214 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:27.242752 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:27.243486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:27.250450 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:27.250967 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:27.251407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:27.252337 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:27.252810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:27.253243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:27.321848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:27.331255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-dbbd933c-cdf2-4134-b4be-a7878d4f12f5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1266 microversion: 2.1 time: 0.093147 Sep 18 01:09:27.331823 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 533/1061] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:27 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1266 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:28.353993 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:28.354625 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:28.362755 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:28.364496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:28.364496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:28.364496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:28.365013 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:28.365278 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:28.506659 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:28.517938 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-68fc707a-f6c1-4354-9f7e-8e435a80f53b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1314 microversion: 2.1 time: 0.167420 Sep 18 01:09:28.518784 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 529/1062] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:28 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1314 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:28.733386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-8a3f3468-9904-48ee-9940-ce18946a333b req-3a64b792-d56a-4d45-a44e-18fe3bc9675a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d24d991f-0b33-485d-b69a-141e7495d514", "tag": "1e4424a5-e1da-41c9-9944-f7e928cf4dd4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:28.744648 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-8a3f3468-9904-48ee-9940-ce18946a333b req-3a64b792-d56a-4d45-a44e-18fe3bc9675a service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:28.745127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-8a3f3468-9904-48ee-9940-ce18946a333b req-3a64b792-d56a-4d45-a44e-18fe3bc9675a service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:28.745562 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-8a3f3468-9904-48ee-9940-ce18946a333b req-3a64b792-d56a-4d45-a44e-18fe3bc9675a service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:28.757848 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-8a3f3468-9904-48ee-9940-ce18946a333b req-3a64b792-d56a-4d45-a44e-18fe3bc9675a service nova] Creating event network-changed:1e4424a5-e1da-41c9-9944-f7e928cf4dd4 for instance d24d991f-0b33-485d-b69a-141e7495d514 on hv-cinder-maste Sep 18 01:09:28.762971 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-8a3f3468-9904-48ee-9940-ce18946a333b req-3a64b792-d56a-4d45-a44e-18fe3bc9675a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033252 Sep 18 01:09:28.763561 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 534/1063] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:29.541282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:29.541929 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:29.548589 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:29.549077 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:29.549515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:29.550216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:29.550704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:29.551136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:29.686560 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:29.695137 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-4581fdee-0415-4bd2-8762-dd8a29617e5e tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1314 microversion: 2.1 time: 0.157698 Sep 18 01:09:29.695717 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 530/1064] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:29 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:30.718170 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:30.719159 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:30.725334 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:30.725822 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:30.726261 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:30.726984 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:30.727434 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:30.727859 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:30.857555 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:30.866060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e89c3f04-723e-4448-87b0-b8536486d1dd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1314 microversion: 2.1 time: 0.152524 Sep 18 01:09:30.866681 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 535/1065] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:30 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:31.888834 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:31.889432 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:31.896512 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:31.897010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:31.897426 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:31.898119 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:31.898597 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:31.899017 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:32.036788 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:32.044785 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fd6476b7-de97-4154-9a5c-de6d6ba99a9d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1314 microversion: 2.1 time: 0.159814 Sep 18 01:09:32.045358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 531/1066] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:31 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:33.070590 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:33.071201 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:33.077576 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:33.078060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:33.078521 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:33.079726 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:33.080184 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:33.080609 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:33.210442 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:33.218100 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-0b1672da-1d37-41f2-a228-17e6a8980cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1314 microversion: 2.1 time: 0.151477 Sep 18 01:09:33.218949 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 536/1067] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:33 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:34.241161 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:34.241771 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:34.248069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:34.248549 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:34.248984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:34.249669 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:34.250124 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:34.250573 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:34.379486 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:34.387241 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b3568596-9b9c-437b-95a0-4f477e571d02 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1314 microversion: 2.1 time: 0.149934 Sep 18 01:09:34.387818 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 532/1068] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:34 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:34.508005 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-e9069299-5a9d-43da-9969-f8a86640d26e req-5e5d96e9-4094-4cf0-9fce-2c07424de7c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d24d991f-0b33-485d-b69a-141e7495d514", "name": "network-vif-plugged", "status": "completed", "tag": "1e4424a5-e1da-41c9-9944-f7e928cf4dd4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:34.520021 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e9069299-5a9d-43da-9969-f8a86640d26e req-5e5d96e9-4094-4cf0-9fce-2c07424de7c4 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:34.521646 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e9069299-5a9d-43da-9969-f8a86640d26e req-5e5d96e9-4094-4cf0-9fce-2c07424de7c4 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:34.522095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-e9069299-5a9d-43da-9969-f8a86640d26e req-5e5d96e9-4094-4cf0-9fce-2c07424de7c4 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:34.535072 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-e9069299-5a9d-43da-9969-f8a86640d26e req-5e5d96e9-4094-4cf0-9fce-2c07424de7c4 service nova] Creating event network-vif-plugged:1e4424a5-e1da-41c9-9944-f7e928cf4dd4 for instance d24d991f-0b33-485d-b69a-141e7495d514 on hv-cinder-maste Sep 18 01:09:34.540127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-e9069299-5a9d-43da-9969-f8a86640d26e req-5e5d96e9-4094-4cf0-9fce-2c07424de7c4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035888 Sep 18 01:09:34.540720 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 537/1069] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:35.411609 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:35.412231 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:35.418750 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:35.419240 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:35.419661 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:35.420343 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:35.420789 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:35.421207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:35.555016 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:35.562703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-516a1fa1-e310-425e-8406-bbf764558e7f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1314 microversion: 2.1 time: 0.155522 Sep 18 01:09:35.563270 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 533/1070] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:35 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:36.585684 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:36.586374 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:36.588794 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:36.589475 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=6 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:36.597871 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:36.598373 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:36.598860 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:36.599557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:36.600003 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:36.600477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:36.727919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:36.735501 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-a4e22470-548e-452a-bb18-75efb6494aa5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1498 microversion: 2.1 time: 0.154066 Sep 18 01:09:36.736060 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 538/1071] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:36 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1498 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:37.237102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:37.237622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:37.239485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:37.245576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:37.246057 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:37.246515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:37.247204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:37.247654 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:37.248074 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:37.273836 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:09:37.348927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-42e2d4dd-3ce5-4e9e-b1bc-1aa8c0e9f23a tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 204 len: 0 microversion: 2.1 time: 0.115388 Sep 18 01:09:37.349619 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 534/1072] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:09:37 2022] DELETE /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:09:37.358054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:37.358698 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:37.364783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:37.365263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:37.365697 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:37.366378 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:37.366854 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:37.367274 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:37.502554 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:37.510978 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-33a254d3-ca60-42d8-bd8e-2979049394a8 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1504 microversion: 2.1 time: 0.156657 Sep 18 01:09:37.511555 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 539/1073] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:37 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1504 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:38.536888 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:38.537591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:38.544063 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:38.544542 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:38.544979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:38.545676 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:38.546131 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:38.546591 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:38.681273 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:38.689231 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e99a3be6-616c-4e4b-a96f-ed768ad25061 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1504 microversion: 2.1 time: 0.157860 Sep 18 01:09:38.689873 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 535/1074] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:38 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1504 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:39.423793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-af3568ee-7cc3-4cf0-8f73-e11e0fb28636 req-cebdf880-3cf6-4b8b-9b12-84ccb0034c73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d24d991f-0b33-485d-b69a-141e7495d514", "name": "network-vif-deleted", "tag": "1e4424a5-e1da-41c9-9944-f7e928cf4dd4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:39.434605 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-af3568ee-7cc3-4cf0-8f73-e11e0fb28636 req-cebdf880-3cf6-4b8b-9b12-84ccb0034c73 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:39.435084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-af3568ee-7cc3-4cf0-8f73-e11e0fb28636 req-cebdf880-3cf6-4b8b-9b12-84ccb0034c73 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:39.435517 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-af3568ee-7cc3-4cf0-8f73-e11e0fb28636 req-cebdf880-3cf6-4b8b-9b12-84ccb0034c73 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:39.447494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-af3568ee-7cc3-4cf0-8f73-e11e0fb28636 req-cebdf880-3cf6-4b8b-9b12-84ccb0034c73 service nova] Creating event network-vif-deleted:1e4424a5-e1da-41c9-9944-f7e928cf4dd4 for instance d24d991f-0b33-485d-b69a-141e7495d514 on hv-cinder-maste Sep 18 01:09:39.452989 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-af3568ee-7cc3-4cf0-8f73-e11e0fb28636 req-cebdf880-3cf6-4b8b-9b12-84ccb0034c73 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032857 Sep 18 01:09:39.453829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 540/1075] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:39.712655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:39.713260 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:39.719895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:39.720390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:39.720803 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:39.722094 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:39.722570 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:39.722996 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:39.753144 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:39.753620 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:39.754050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:39.754515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lazy-loading 'fault' on Instance uuid d24d991f-0b33-485d-b69a-141e7495d514 {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:09:39.795711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:39.803626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-6dd224c7-7ec0-43c7-8b36-52dd5884fccd tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 200 len: 1304 microversion: 2.1 time: 0.094775 Sep 18 01:09:39.804210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 536/1076] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:39 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 1304 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:09:40.829099 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:40.829810 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: d24d991f-0b33-485d-b69a-141e7495d514] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:40.841610 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:40.842642 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:40.843113 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:40.844091 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:40.844559 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:40.846585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:40.860796 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] HTTP exception thrown: Instance d24d991f-0b33-485d-b69a-141e7495d514 could not be found. Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Request method failure captured: Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 HTTP/1.1 Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm_EAYoWdLaA5EgjYRdLHtlqfUT0XaCZFoevWkXTrNcmbbfoSq4nXxIX4seDrHoeHbmhuM352EfCRlyWxsc0uCeAUk3znAxHm8ODmR_sOf1jQqqdj6Xd0BWtsQzuRovoYnFnHpLafmPywrVOAEzQuMtsW0h7v8rvDdrmiFd1y6EosFo Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: c83778305ed1494f931ca66d197fa5d3 Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesActionsTest-849219088 Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:09:40.863086 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c83778305ed1494f931ca66d197fa5d3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c83778305ed1494f931ca66d197fa5d3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c83778305ed1494f931ca66d197fa5d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c83778305ed1494f931ca66d197fa5d3"}]}] Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesActionsTest-849219088 Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: c83778305ed1494f931ca66d197fa5d3 Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesActionsTest-849219088 Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesActionsTest-849219088-project Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 33ea64ca8ec14dd2a34a05ce90f9fa7a Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesActionsTest-849219088-project Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance d24d991f-0b33-485d-b69a-141e7495d514 could not be found. Sep 18 01:09:40.864585 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': 'd24d991f-0b33-485d-b69a-141e7495d514'} Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d24d991f-0b33-485d-b69a-141e7495d514 could not be found. Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:09:40.865496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d24d991f-0b33-485d-b69a-141e7495d514 could not be found. Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d24d991f-0b33-485d-b69a-141e7495d514 could not be found. Sep 18 01:09:40.867235 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:40.868765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Returning 404 to user: Instance d24d991f-0b33-485d-b69a-141e7495d514 could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:09:40.868765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c1c5dcf7-77a0-4a44-ab8f-a7c74acd805c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514" status: 404 len: 111 microversion: 2.1 time: 0.040360 Sep 18 01:09:40.868765 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 541/1077] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:40 2022] GET /compute/v2.1/servers/d24d991f-0b33-485d-b69a-141e7495d514 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:09:40.914792 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1352319606", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "networks": [{"uuid": "d37bfa6d-c1b4-4c46-b4de-962869a5a79e"}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:40.969741 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] validate_networks() for [('d37bfa6d-c1b4-4c46-b4de-962869a5a79e', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:09:41.379010 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:09:41.379433 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:09:41.379931 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:09:41.380338 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:09:41.380748 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:09:41.389454 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:41.389932 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:41.390366 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:41.391123 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:41.391576 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:41.391997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:41.408923 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Getting quotas for project c83778305ed1494f931ca66d197fa5d3. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:09:41.412934 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Getting quotas for user 33ea64ca8ec14dd2a34a05ce90f9fa7a and project c83778305ed1494f931ca66d197fa5d3. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:09:41.424033 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:09:41.426322 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:09:41.473723 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7e2616b3-768e-40c0-9df6-e8c487da282c tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.562929 Sep 18 01:09:41.474474 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 537/1078] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:09:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 564 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:09:41.485489 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-94d1c2f4-ddfb-4d8e-833a-a0fbfc022cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:41.486112 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-94d1c2f4-ddfb-4d8e-833a-a0fbfc022cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:41.552435 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-94d1c2f4-ddfb-4d8e-833a-a0fbfc022cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:41.559259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-94d1c2f4-ddfb-4d8e-833a-a0fbfc022cdb tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1214 microversion: 2.1 time: 0.077691 Sep 18 01:09:41.559821 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 542/1079] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:41 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1214 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:42.584170 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:42.584809 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:42.591688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:42.592175 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:42.592613 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:42.593308 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:42.594041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:42.594490 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:42.660931 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:42.671790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7020c40b-68a4-4e0d-849f-52331d794d00 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1274 microversion: 2.1 time: 0.092367 Sep 18 01:09:42.672383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 538/1080] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:42 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1274 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:43.694770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:43.695440 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:43.702678 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:43.703433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:43.703738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:43.704435 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:43.704885 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:43.705303 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:43.837133 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:43.845176 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e2bfe8fe-27c5-4599-97a3-68b02aceb4c9 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1314 microversion: 2.1 time: 0.154351 Sep 18 01:09:43.845830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 543/1081] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:43 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:44.026166 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-1995115f-d2be-4bf7-ad27-1196fd185135 req-70086004-fbba-4f19-9f95-a4be7e444f98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e", "tag": "4f169f34-6967-414d-a438-cec65ebc920d"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:44.037127 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-1995115f-d2be-4bf7-ad27-1196fd185135 req-70086004-fbba-4f19-9f95-a4be7e444f98 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:44.037613 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-1995115f-d2be-4bf7-ad27-1196fd185135 req-70086004-fbba-4f19-9f95-a4be7e444f98 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:44.038039 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-1995115f-d2be-4bf7-ad27-1196fd185135 req-70086004-fbba-4f19-9f95-a4be7e444f98 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:44.050116 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-1995115f-d2be-4bf7-ad27-1196fd185135 req-70086004-fbba-4f19-9f95-a4be7e444f98 service nova] Creating event network-changed:4f169f34-6967-414d-a438-cec65ebc920d for instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e on hv-cinder-maste Sep 18 01:09:44.057069 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-1995115f-d2be-4bf7-ad27-1196fd185135 req-70086004-fbba-4f19-9f95-a4be7e444f98 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034577 Sep 18 01:09:44.057667 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 539/1082] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:09:44.868845 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:44.869467 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:44.875934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:44.876408 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:44.876835 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:44.878114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:44.878582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:44.878999 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:45.013770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:45.021253 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8ec66a2f-ee46-4950-a98c-bd1fff4bc32d tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1314 microversion: 2.1 time: 0.156323 Sep 18 01:09:45.021799 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 544/1083] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:44 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:46.044812 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:46.045723 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:46.052006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:46.052494 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:46.052932 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:46.053632 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:46.054086 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:46.055176 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:46.191673 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:46.202855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-ec0a7330-cd54-4890-b459-3794d0795e90 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1314 microversion: 2.1 time: 0.162561 Sep 18 01:09:46.203766 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 540/1084] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:46 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:47.226388 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:47.227020 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:47.233391 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:47.233870 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:47.234302 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:47.235014 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:47.235464 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:47.235879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:47.361554 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:47.369644 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-4fe6f255-3f51-4de2-9a21-9861d5d68066 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1314 microversion: 2.1 time: 0.146948 Sep 18 01:09:47.370254 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 545/1085] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:47 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1314 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:48.393798 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:48.394415 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:48.401396 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:48.401893 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:48.402363 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:48.403118 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:48.403711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:48.404087 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:48.519178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-77487f7b-9f3d-4c64-b168-b992c388a697 req-33b04dbc-5e71-4dd3-a670-a9a6657328e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e", "name": "network-vif-plugged", "status": "completed", "tag": "4f169f34-6967-414d-a438-cec65ebc920d"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:48.532080 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-77487f7b-9f3d-4c64-b168-b992c388a697 req-33b04dbc-5e71-4dd3-a670-a9a6657328e8 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:48.533381 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-77487f7b-9f3d-4c64-b168-b992c388a697 req-33b04dbc-5e71-4dd3-a670-a9a6657328e8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:48.534408 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-77487f7b-9f3d-4c64-b168-b992c388a697 req-33b04dbc-5e71-4dd3-a670-a9a6657328e8 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:48.547223 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-77487f7b-9f3d-4c64-b168-b992c388a697 req-33b04dbc-5e71-4dd3-a670-a9a6657328e8 service nova] Creating event network-vif-plugged:4f169f34-6967-414d-a438-cec65ebc920d for instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e on hv-cinder-maste Sep 18 01:09:48.551943 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:48.552970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-77487f7b-9f3d-4c64-b168-b992c388a697 req-33b04dbc-5e71-4dd3-a670-a9a6657328e8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038001 Sep 18 01:09:48.553876 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 546/1086] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 18 01:09:48.560850 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-aed02d27-a49f-4623-b6a4-3dd432bced83 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1314 microversion: 2.1 time: 0.170912 Sep 18 01:09:48.561729 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 541/1087] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:48 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:49.583386 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:49.584054 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:49.590716 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:49.591191 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:49.591619 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:49.592298 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:49.592746 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:49.593164 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:49.722915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:49.730548 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1eb5280b-df09-4389-8ade-16667922de56 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1314 microversion: 2.1 time: 0.151360 Sep 18 01:09:49.731114 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 547/1088] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:49 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:50.754482 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:50.755047 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:50.761442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:50.761922 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:50.762358 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:50.763074 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:50.763528 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:50.763951 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:50.893927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:50.901685 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-b3f57f58-3d87-43ed-a8a1-f15001e19b1f tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1497 microversion: 2.1 time: 0.151850 Sep 18 01:09:50.902430 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 542/1089] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:50 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:51.442802 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:51.443328 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:51.449700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:51.450186 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:51.450654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:51.451374 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:51.451830 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:51.452267 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:51.479271 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:09:51.557221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:51.562717 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-c6a710ea-d851-4cf9-831f-e5848fcb20f1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "DELETE /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 204 len: 0 microversion: 2.1 time: 0.123214 Sep 18 01:09:51.563545 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 548/1090] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:09:51 2022] DELETE /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:09:51.572580 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:51.573197 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:51.575246 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:51.575906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:51.582337 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:51.582858 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:51.583295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:51.584004 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:51.584463 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:51.584889 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:51.714989 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:51.723497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c72427b7-4743-497a-a4d7-b521f57b36d5 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1503 microversion: 2.1 time: 0.155086 Sep 18 01:09:51.724076 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 543/1091] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:51 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1503 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:52.286610 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:52.287259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:52.287963 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-6aa22e17-2e53-4036-a39e-3012129b256f req-ec54d441-6a52-45b2-a664-0a091304a426 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69223fa4-55c0-4c65-b9d4-c2e5ec0b778d", "tag": "ac27aed2-9401-48f5-a764-f83ca47284b4"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:52.294200 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:52.294801 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:52.295301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:52.296065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:52.296595 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:52.297065 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:52.298359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-6aa22e17-2e53-4036-a39e-3012129b256f req-ec54d441-6a52-45b2-a664-0a091304a426 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:52.298870 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-6aa22e17-2e53-4036-a39e-3012129b256f req-ec54d441-6a52-45b2-a664-0a091304a426 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:52.299301 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-6aa22e17-2e53-4036-a39e-3012129b256f req-ec54d441-6a52-45b2-a664-0a091304a426 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:52.311543 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-6aa22e17-2e53-4036-a39e-3012129b256f req-ec54d441-6a52-45b2-a664-0a091304a426 service nova] Creating event network-changed:ac27aed2-9401-48f5-a764-f83ca47284b4 for instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d on hv-cinder-maste Sep 18 01:09:52.317574 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-6aa22e17-2e53-4036-a39e-3012129b256f req-ec54d441-6a52-45b2-a664-0a091304a426 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033154 Sep 18 01:09:52.318168 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 544/1092] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:52.328794 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:09:52.404646 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-d04a6e5a-441d-420d-8842-8e2a70fd75f3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "DELETE /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 204 len: 0 microversion: 2.1 time: 0.121885 Sep 18 01:09:52.405263 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 549/1093] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:09:52 2022] DELETE /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:09:52.414096 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:52.414876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:52.420949 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:52.421424 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:52.421855 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:52.422561 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:52.423012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:52.423579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:52.442722 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:52.565672 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:52.573560 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e9cf355a-1c89-4595-b729-7f0b1bea0695 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.163886 Sep 18 01:09:52.574129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 545/1094] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:52 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:52.746653 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:52.747361 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:52.753934 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:52.754424 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:52.754889 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:52.756108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:52.756108 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:52.756485 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:52.761084 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:52.897111 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:52.905781 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-374fadb1-30aa-43e1-9243-d29b299244e1 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1503 microversion: 2.1 time: 0.163379 Sep 18 01:09:52.906539 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 550/1095] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:52 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1503 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:53.597394 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:53.597998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:53.604600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:53.604998 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:53.605442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:53.606159 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:53.606601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:53.607024 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:53.744530 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:53.755776 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-90d494bd-d5d1-41f8-8eae-6d803f8b6c48 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.162162 Sep 18 01:09:53.756364 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 546/1096] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:53 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:53.931193 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:53.931877 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:53.938148 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:53.938666 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:53.939097 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:53.939774 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:53.940225 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:53.940643 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:54.079007 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:54.086908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-3c00f056-a265-42aa-ae35-5f71df4ab0f7 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1503 microversion: 2.1 time: 0.159921 Sep 18 01:09:54.087469 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 551/1097] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:53 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:54.780453 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:54.781072 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:54.789857 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:54.790242 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:54.790712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:54.791637 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:54.792102 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:54.792532 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:55.010994 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:55.022104 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-fc9bb61a-aac8-42db-b725-1a1136d9f204 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1568 microversion: 2.1 time: 0.246724 Sep 18 01:09:55.035198 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 547/1098] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:54 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1568 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:55.111352 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:55.112612 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:55.118639 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:55.119483 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:55.119914 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:55.120823 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:55.121279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:55.121711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:55.238597 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '33ea64ca8ec14dd2a34a05ce90f9fa7a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c83778305ed1494f931ca66d197fa5d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:55.247328 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-07b881ca-7e2d-407d-843a-22f3748e302b tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 200 len: 1461 microversion: 2.1 time: 0.140200 Sep 18 01:09:55.247951 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 552/1099] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:55 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 1461 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:55.281860 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-911e4c92-c84c-484f-adab-9a6aa8288f5c req-9c71628f-9566-405e-ab93-265904b444e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e", "name": "network-vif-deleted", "tag": "4f169f34-6967-414d-a438-cec65ebc920d"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:55.298307 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-911e4c92-c84c-484f-adab-9a6aa8288f5c req-9c71628f-9566-405e-ab93-265904b444e4 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:55.299085 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-911e4c92-c84c-484f-adab-9a6aa8288f5c req-9c71628f-9566-405e-ab93-265904b444e4 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:55.299516 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-911e4c92-c84c-484f-adab-9a6aa8288f5c req-9c71628f-9566-405e-ab93-265904b444e4 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:55.311299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-911e4c92-c84c-484f-adab-9a6aa8288f5c req-9c71628f-9566-405e-ab93-265904b444e4 service nova] Creating event network-vif-deleted:4f169f34-6967-414d-a438-cec65ebc920d for instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e on hv-cinder-maste Sep 18 01:09:55.316610 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-911e4c92-c84c-484f-adab-9a6aa8288f5c req-9c71628f-9566-405e-ab93-265904b444e4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038005 Sep 18 01:09:55.317419 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 548/1100] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:55.618433 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-20175cbd-e6a6-4fef-8ceb-b280762cb96c req-73e6a561-f323-4f6a-9995-7e819bae9775 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69223fa4-55c0-4c65-b9d4-c2e5ec0b778d", "name": "network-vif-deleted", "tag": "ac27aed2-9401-48f5-a764-f83ca47284b4"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:09:55.627691 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-20175cbd-e6a6-4fef-8ceb-b280762cb96c req-73e6a561-f323-4f6a-9995-7e819bae9775 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:55.628163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-20175cbd-e6a6-4fef-8ceb-b280762cb96c req-73e6a561-f323-4f6a-9995-7e819bae9775 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:55.629226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-20175cbd-e6a6-4fef-8ceb-b280762cb96c req-73e6a561-f323-4f6a-9995-7e819bae9775 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:55.639866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-20175cbd-e6a6-4fef-8ceb-b280762cb96c req-73e6a561-f323-4f6a-9995-7e819bae9775 service nova] Creating event network-vif-deleted:ac27aed2-9401-48f5-a764-f83ca47284b4 for instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d on hv-cinder-maste Sep 18 01:09:55.644693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-20175cbd-e6a6-4fef-8ceb-b280762cb96c req-73e6a561-f323-4f6a-9995-7e819bae9775 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029981 Sep 18 01:09:55.645260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 553/1101] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:09:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:09:56.054179 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:56.055511 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:56.066451 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:56.067325 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:56.068135 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:56.069405 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:56.070031 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:56.070839 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:56.141339 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ea84c1e0dcf441fb1346eaaef4de421', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2d812ad5fce24a49b0f2d07081d5718a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:09:56.149412 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-eb1984a6-6277-4c05-8cb7-d6c7ee6af746 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 200 len: 1220 microversion: 2.1 time: 0.103621 Sep 18 01:09:56.149925 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 549/1102] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:56 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 1220 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:09:56.273626 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:56.274918 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] [instance: 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:56.277291 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:56.283537 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:56.284183 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:56.285178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:56.286073 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:56.287210 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:56.289025 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:56.305431 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] HTTP exception thrown: Instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e could not be found. Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Request method failure captured: Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e HTTP/1.1 Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm_EAYoWdLaA5EgjYRdLHtlqfUT0XaCZFoevWkXTrNcmbbfoSq4nXxIX4seDrHoeHbmhuM352EfCRlyWxsc0uCeAUk3znAxHm8ODmR_sOf1jQqqdj6Xd0BWtsQzuRovoYnFnHpLafmPywrVOAEzQuMtsW0h7v8rvDdrmiFd1y6EosFo Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: c83778305ed1494f931ca66d197fa5d3 Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesActionsTest-849219088 Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:09:56.307152 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c83778305ed1494f931ca66d197fa5d3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c83778305ed1494f931ca66d197fa5d3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c83778305ed1494f931ca66d197fa5d3"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c83778305ed1494f931ca66d197fa5d3"}]}] Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesActionsTest-849219088 Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: c83778305ed1494f931ca66d197fa5d3 Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesActionsTest-849219088 Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesActionsTest-849219088-project Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: 33ea64ca8ec14dd2a34a05ce90f9fa7a Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesActionsTest-849219088-project Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e could not be found. Sep 18 01:09:56.308699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e'} Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e could not be found. Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:09:56.309514 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e could not be found. Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e could not be found. Sep 18 01:09:56.310842 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:09:56.312015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] Returning 404 to user: Instance 04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:09:56.312015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-8dcd5467-77dd-493e-b261-55b3163e4508 tempest-VolumesActionsTest-849219088 tempest-VolumesActionsTest-849219088-project] 10.222.0.42 "GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e" status: 404 len: 111 microversion: 2.1 time: 0.039127 Sep 18 01:09:56.312015 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 554/1103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:56 2022] GET /compute/v2.1/servers/04ff6e3b-fd58-49d7-a5a6-f9d97ce8f66e => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:09:57.172650 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:57.173177 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] [instance: 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:09:57.175661 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:09:57.180816 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:57.181298 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:57.181737 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:57.182423 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:09:57.182906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:09:57.183329 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:09:57.197831 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] HTTP exception thrown: Instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d could not be found. Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Request method failure captured: Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d HTTP/1.1 Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm7LTfRSgsuBaHv1IEkReppEBwIcBqMoKFvL96GjnjeiNweXWmfPWawvplgzFaHVShheuxp0x2T07DNtmKkfvfNTf20lgpe1yEomijWR02bBU5_JaE9i0-iFz64Kp3AyyWbN_UdtIdSvKlfe6Eh-YxLBQsjyOn0EGDz7DMcF69gPoMY Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: 2d812ad5fce24a49b0f2d07081d5718a Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-TestShelveInstance-650726549 Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:09:57.199496 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2d812ad5fce24a49b0f2d07081d5718a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2d812ad5fce24a49b0f2d07081d5718a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2d812ad5fce24a49b0f2d07081d5718a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2d812ad5fce24a49b0f2d07081d5718a"}]}] Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-TestShelveInstance-650726549 Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: 2d812ad5fce24a49b0f2d07081d5718a Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-TestShelveInstance-650726549 Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-TestShelveInstance-650726549-project Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: 7ea84c1e0dcf441fb1346eaaef4de421 Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-TestShelveInstance-650726549-project Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d could not be found. Sep 18 01:09:57.201163 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': '69223fa4-55c0-4c65-b9d4-c2e5ec0b778d'} Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d could not be found. Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:09:57.202388 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d could not be found. Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d could not be found. Sep 18 01:09:57.203954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:09:57.205503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Returning 404 to user: Instance 69223fa4-55c0-4c65-b9d4-c2e5ec0b778d could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:09:57.205503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e4cc4b1b-5b4b-4061-a520-c47c2c2b36a3 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d" status: 404 len: 111 microversion: 2.1 time: 0.032166 Sep 18 01:09:57.205503 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 550/1104] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:09:57 2022] GET /compute/v2.1/servers/69223fa4-55c0-4c65-b9d4-c2e5ec0b778d => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:09:57.363710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1820d57b-a525-473c-8f24-d114dd19e894 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] Calling method '.version_select at 0x7fcab1d4be50>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:09:57.387479 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1820d57b-a525-473c-8f24-d114dd19e894 tempest-TestShelveInstance-650726549 tempest-TestShelveInstance-650726549-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1667373744" status: 202 len: 0 microversion: 2.1 time: 0.027439 Sep 18 01:09:57.387898 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 555/1105] 10.222.0.42 () {58 vars in 1254 bytes} [Sun Sep 18 01:09:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1667373744 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:10:22.153117 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1166547488", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "networks": [{"uuid": "570e1a4b-dc74-4368-927f-9538b7aee5a3"}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:10:22.162528 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 18 01:10:22.162677 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Sun Sep 18 01:10:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.42) !!! Sep 18 01:10:22.163655 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=24 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:22.164301 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:22.211520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] validate_networks() for [('570e1a4b-dc74-4368-927f-9538b7aee5a3', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:10:22.552393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:10:22.552823 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:10:22.553299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:10:22.553712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:10:22.554125 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:10:22.563091 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:22.563574 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:22.564012 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:22.564752 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:22.565210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:22.565643 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:22.583322 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Getting quotas for project a4f90e9a8c164448a214481704ff18a2. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:10:22.587253 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Getting quotas for user cbfab0fe9b4f4840a3ad9953d90e86bb and project a4f90e9a8c164448a214481704ff18a2. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:10:22.599084 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:10:22.601349 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:10:22.649578 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5506787f-9f38-4d67-a242-94650ff35dec tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.500287 Sep 18 01:10:22.650282 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 551/1106] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:10:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 502 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:10:22.659012 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=20 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:22.659959 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=4,nova_api:DELETE=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:22.665230 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-1ce653f3-7238-44de-9e3b-886be7bdb4fb tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:22.665866 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-1ce653f3-7238-44de-9e3b-886be7bdb4fb tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:22.722387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-1ce653f3-7238-44de-9e3b-886be7bdb4fb tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:22.729215 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-1ce653f3-7238-44de-9e3b-886be7bdb4fb tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1219 microversion: 2.1 time: 0.071687 Sep 18 01:10:22.729771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 556/1107] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:22 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1219 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:10:23.752204 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:23.752805 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:23.760688 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:23.761171 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:23.761602 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:23.762295 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:23.762773 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:23.763193 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:23.828485 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:23.837505 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-8d9a019a-8bef-41e8-893b-620b0c8f4846 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1279 microversion: 2.1 time: 0.089154 Sep 18 01:10:23.838390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 552/1108] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:23 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1279 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:24.867775 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:24.868389 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:24.874741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:24.875229 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:24.875667 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:24.876358 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:24.876813 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:24.877348 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:25.010754 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:25.018364 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-19cfa93f-187e-4bd0-80be-4df56d87aa3f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1319 microversion: 2.1 time: 0.154467 Sep 18 01:10:25.019679 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 557/1109] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:24 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1319 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:25.182613 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-accf1000-2dde-4e20-959c-a719d46bb365 req-d2b0875c-fd15-4a10-abad-35dd7981cbb6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1239ceb1-692b-4ef8-919f-8f8aed47d0cb", "tag": "05a2f546-bfc0-431b-b8d2-43fe9120fd1e"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:10:25.193772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-accf1000-2dde-4e20-959c-a719d46bb365 req-d2b0875c-fd15-4a10-abad-35dd7981cbb6 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:25.194258 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-accf1000-2dde-4e20-959c-a719d46bb365 req-d2b0875c-fd15-4a10-abad-35dd7981cbb6 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:25.194714 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-accf1000-2dde-4e20-959c-a719d46bb365 req-d2b0875c-fd15-4a10-abad-35dd7981cbb6 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:25.208101 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-accf1000-2dde-4e20-959c-a719d46bb365 req-d2b0875c-fd15-4a10-abad-35dd7981cbb6 service nova] Creating event network-changed:05a2f546-bfc0-431b-b8d2-43fe9120fd1e for instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb on hv-cinder-maste Sep 18 01:10:25.212941 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-accf1000-2dde-4e20-959c-a719d46bb365 req-d2b0875c-fd15-4a10-abad-35dd7981cbb6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034100 Sep 18 01:10:25.213531 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 553/1110] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:10:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:10:26.041937 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:26.042557 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:26.048923 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:26.049395 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:26.049825 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:26.051213 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:26.051690 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:26.052078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:26.180710 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:26.188202 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-24f1f09f-5c32-45a4-9d34-a157184c1623 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1319 microversion: 2.1 time: 0.150133 Sep 18 01:10:26.188762 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 558/1111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:26 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1319 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:27.211023 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:27.211726 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:27.217927 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:27.218409 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:27.218876 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:27.219569 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:27.220025 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:27.220709 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:27.354745 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:27.362555 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-f4658d33-fe34-4cb4-ab9e-da490c4859a9 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1319 microversion: 2.1 time: 0.156230 Sep 18 01:10:27.363607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 554/1112] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:27 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:28.384886 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:28.385492 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:28.392056 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:28.392535 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:28.392970 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:28.393654 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:28.394387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:28.394897 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:28.528353 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:28.536817 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-99a83507-5624-44ec-80a3-c3fa1222b4cc tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1319 microversion: 2.1 time: 0.155793 Sep 18 01:10:28.537404 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 559/1113] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:28 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:29.562711 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:29.564091 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:29.574118 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:29.574641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:29.575078 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:29.575772 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:29.576217 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:29.576641 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:29.703045 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:29.711107 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3d593c9d-0d51-4a35-8888-4cef344a15ce tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1319 microversion: 2.1 time: 0.153727 Sep 18 01:10:29.711672 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 555/1114] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:29 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:30.225346 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-5eacfe1d-3a11-4403-a8b8-cf0062887e16 req-1a3f96af-5248-4933-8a5b-be83e5019b85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1239ceb1-692b-4ef8-919f-8f8aed47d0cb", "name": "network-vif-plugged", "status": "completed", "tag": "05a2f546-bfc0-431b-b8d2-43fe9120fd1e"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:10:30.236243 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-5eacfe1d-3a11-4403-a8b8-cf0062887e16 req-1a3f96af-5248-4933-8a5b-be83e5019b85 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:30.236695 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-5eacfe1d-3a11-4403-a8b8-cf0062887e16 req-1a3f96af-5248-4933-8a5b-be83e5019b85 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:30.237130 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-5eacfe1d-3a11-4403-a8b8-cf0062887e16 req-1a3f96af-5248-4933-8a5b-be83e5019b85 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:30.248255 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-5eacfe1d-3a11-4403-a8b8-cf0062887e16 req-1a3f96af-5248-4933-8a5b-be83e5019b85 service nova] Creating event network-vif-plugged:05a2f546-bfc0-431b-b8d2-43fe9120fd1e for instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb on hv-cinder-maste Sep 18 01:10:30.253259 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-5eacfe1d-3a11-4403-a8b8-cf0062887e16 req-1a3f96af-5248-4933-8a5b-be83e5019b85 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032697 Sep 18 01:10:30.253841 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 560/1115] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:10:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:10:30.734331 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:30.734947 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:30.741370 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:30.741847 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:30.742281 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:30.742997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:30.743450 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:30.743871 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:31.120194 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:31.127764 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e1d4fdfb-1496-47bf-ab97-fbc6fd13809c tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1319 microversion: 2.1 time: 0.397392 Sep 18 01:10:31.128960 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 556/1116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:30 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1319 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:32.152380 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:10:32.153078 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:32.159290 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:32.159771 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:32.160203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:32.160894 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:32.161358 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:32.161766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:32.288473 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:10:32.296228 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-6c1db5af-3efb-48b2-891f-6445032320b1 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1509 microversion: 2.1 time: 0.148601 Sep 18 01:10:32.296793 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 561/1117] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:10:32 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1509 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:10:46.445120 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0c9bca05-c097-4f37-9451-082d719743b8", "device": "/dev/sdb"}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:10:46.448157 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:10:46.450320 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:46.451000 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:46.451848 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=23,nova_api:INSERT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:46.452513 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=28 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:10:46.459259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:46.459754 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:46.460194 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:46.460902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:10:46.461359 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:10:46.461788 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:10:46.723050 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5122f062-a632-4064-985b-c3b0b60ca1e5 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "POST /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.281854 Sep 18 01:10:46.723703 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 557/1118] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:10:46 2022] POST /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb/os-volume_attachments => generated 194 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:11:10.414432 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:10.414981 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:10.417116 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:10.417829 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=14 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:10.418496 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=26 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:10.425770 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:10.426258 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:10.426731 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:10.427944 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:10.428399 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:10.428828 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:10.508790 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.objects.instance [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lazy-loading 'services' on Instance uuid 1239ceb1-692b-4ef8-919f-8f8aed47d0cb {{(pid=120831) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:11:10.577640 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-80a9a3bd-a769-48bf-8c6a-ffe882f71001 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb/os-volume_attachments/0c9bca05-c097-4f37-9451-082d719743b8" status: 202 len: 0 microversion: 2.1 time: 0.167065 Sep 18 01:11:10.578224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 562/1119] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:11:10 2022] DELETE /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb/os-volume_attachments/0c9bca05-c097-4f37-9451-082d719743b8 => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:11:11.870008 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:11.870896 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:11.875762 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:11.876375 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:11.884428 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:11.884906 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:11.885343 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:11.886041 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:11.886515 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:11.886953 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:11.913393 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Going to try to terminate instance {{(pid=120832) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:11:11.985174 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-3fb03038-8868-44a8-a8b5-c330c8f2d134 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 204 len: 0 microversion: 2.1 time: 0.116302 Sep 18 01:11:11.985796 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 558/1120] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:11:11 2022] DELETE /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:11:11.995217 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:11.995912 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:12.001988 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:12.002486 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:12.002920 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:12.003599 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:12.004055 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:12.004459 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:12.136426 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:12.144356 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-65da29e8-d8d0-4439-8538-9f35eb88593b tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1515 microversion: 2.1 time: 0.153804 Sep 18 01:11:12.144915 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 563/1121] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:11 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1515 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:13.168818 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:13.169509 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:13.175830 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:13.176307 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:13.176734 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:13.178064 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:13.178538 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:13.178959 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:13.302526 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:13.310335 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-35190ad2-51ec-4cd0-9fa8-a7c46a3ecfcd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1515 microversion: 2.1 time: 0.145798 Sep 18 01:11:13.310944 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 559/1122] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:13 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1515 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:14.254361 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-1fdacba3-3410-4d39-91cd-2867a9843f3f req-206441e2-9853-41e9-92b8-65129e7a28d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1239ceb1-692b-4ef8-919f-8f8aed47d0cb", "name": "network-vif-deleted", "tag": "05a2f546-bfc0-431b-b8d2-43fe9120fd1e"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:11:14.266685 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-1fdacba3-3410-4d39-91cd-2867a9843f3f req-206441e2-9853-41e9-92b8-65129e7a28d0 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:14.267167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-1fdacba3-3410-4d39-91cd-2867a9843f3f req-206441e2-9853-41e9-92b8-65129e7a28d0 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:14.267600 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-1fdacba3-3410-4d39-91cd-2867a9843f3f req-206441e2-9853-41e9-92b8-65129e7a28d0 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:14.283141 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-1fdacba3-3410-4d39-91cd-2867a9843f3f req-206441e2-9853-41e9-92b8-65129e7a28d0 service nova] Creating event network-vif-deleted:05a2f546-bfc0-431b-b8d2-43fe9120fd1e for instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb on hv-cinder-maste Sep 18 01:11:14.288976 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-1fdacba3-3410-4d39-91cd-2867a9843f3f req-206441e2-9853-41e9-92b8-65129e7a28d0 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038358 Sep 18 01:11:14.289578 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 564/1123] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:11:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:11:14.334133 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:14.334780 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:14.341165 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:14.341715 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:14.342216 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:14.343003 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:14.343520 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:14.344244 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:14.378088 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:14.378600 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:14.379030 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:14.379472 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lazy-loading 'fault' on Instance uuid 1239ceb1-692b-4ef8-919f-8f8aed47d0cb {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:11:14.419656 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:14.428056 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c13b87b1-f3f5-42f8-abc5-f9b11c33bdee tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 200 len: 1309 microversion: 2.1 time: 0.098658 Sep 18 01:11:14.428612 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 560/1124] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:14 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 1309 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:15.451179 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:15.451868 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: 1239ceb1-692b-4ef8-919f-8f8aed47d0cb] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:15.459004 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:15.459477 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:15.459908 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:15.460580 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:15.461027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:15.461446 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:15.475095 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.wsgi [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] HTTP exception thrown: Instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb could not be found. Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Request method failure captured: Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: request: GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb HTTP/1.1 Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept: application/json Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Accept-Encoding: identity Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Connection: close Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Length: 0 Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Content-Type: application/json Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Host: 10.222.0.42 Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: Openstack-System-Scope: None Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: User-Agent: python-urllib3/1.26.12 Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Auth-Token: gAAAAABjJm_8DovFyWm-yip1DbAfFHpStHj6B0b-ZKXPx4ZHj3eBHWyq5Lh56Nb035XnRXCDtYZmn4qlI-tRWf9E8V9pSqAWdzNyXahHmfa9EEwkgEqs86nUnPfyJduYIcJkGL3CYh2NqYvxPudEmqKzjbrx5XGJ23ZXaUkl4sD2wr_aTHmcRhE Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Id: None Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Domain-Name: None Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Identity-Status: Confirmed Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Is-Admin-Project: True Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Id: default Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Domain-Name: Default Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Id: a4f90e9a8c164448a214481704ff18a2 Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1303083818 Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Role: reader,member Sep 18 01:11:15.476693 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Roles: reader,member Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/a4f90e9a8c164448a214481704ff18a2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/a4f90e9a8c164448a214481704ff18a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_a4f90e9a8c164448a214481704ff18a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/a4f90e9a8c164448a214481704ff18a2"}]}] Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant: tempest-VolumesSnapshotTestJSON-1303083818 Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Id: a4f90e9a8c164448a214481704ff18a2 Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1303083818 Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User: tempest-VolumesSnapshotTestJSON-1303083818-project Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Id: default Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Domain-Name: Default Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Id: cbfab0fe9b4f4840a3ad9953d90e86bb Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: X-User-Name: tempest-VolumesSnapshotTestJSON-1303083818-project Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: method: > Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: exception: Instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb could not be found. Sep 18 01:11:15.478616 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: action_args: {'id': '1239ceb1-692b-4ef8-919f-8f8aed47d0cb'} Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb could not be found. Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:11:15.482355 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb could not be found. Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb could not be found. Sep 18 01:11:15.483879 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: ERROR nova.api.openstack.wsgi Sep 18 01:11:15.485158 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Returning 404 to user: Instance 1239ceb1-692b-4ef8-919f-8f8aed47d0cb could not be found. {{(pid=120831) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:11:15.485158 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-e3b99c5e-c512-48ce-9dc9-a00f44362406 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb" status: 404 len: 111 microversion: 2.1 time: 0.031252 Sep 18 01:11:15.485158 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 565/1125] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:15 2022] GET /compute/v2.1/servers/1239ceb1-692b-4ef8-919f-8f8aed47d0cb => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 18 01:11:23.248384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-210681908", "imageRef": "d3d55b64-7833-4bad-93d4-8b9a33b1ef0c", "flavorRef": "452", "networks": [{"uuid": "570e1a4b-dc74-4368-927f-9538b7aee5a3"}]}} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:11:23.302114 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.network.neutron [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] validate_networks() for [('570e1a4b-dc74-4368-927f-9538b7aee5a3', None, None, None, None, None)] {{(pid=120832) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 18 01:11:23.642062 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Flavor limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 18 01:11:23.642517 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Image limits 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 18 01:11:23.642992 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Flavor pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 18 01:11:23.643396 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Image pref 0:0:0 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 18 01:11:23.643802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.virt.hardware [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120832) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 18 01:11:23.653061 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:23.653534 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:23.653965 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:23.654733 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:23.655186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:23.655607 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:23.672219 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Getting quotas for project a4f90e9a8c164448a214481704ff18a2. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 18 01:11:23.676334 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.quota [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Getting quotas for user cbfab0fe9b4f4840a3ad9953d90e86bb and project a4f90e9a8c164448a214481704ff18a2. Resources: {'cores', 'ram', 'instances'} {{(pid=120832) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 18 01:11:23.687601 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Going to run 1 instances... {{(pid=120832) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 18 01:11:23.689835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='d3d55b64-7833-4bad-93d4-8b9a33b1ef0c',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120832) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 18 01:11:23.736398 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c12c7989-98a0-4a35-a057-67e0d4e618fd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.491864 Sep 18 01:11:23.736997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 561/1126] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 18 01:11:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 493 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 18 01:11:23.751301 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ed4dcd21-5ce0-45c9-a22b-06abe40c2f0a tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:23.752083 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ed4dcd21-5ce0-45c9-a22b-06abe40c2f0a tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:23.806013 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ed4dcd21-5ce0-45c9-a22b-06abe40c2f0a tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:23.812856 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ed4dcd21-5ce0-45c9-a22b-06abe40c2f0a tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1218 microversion: 2.1 time: 0.066741 Sep 18 01:11:23.813622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 566/1127] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:23 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1218 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:24.836265 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:24.836941 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:24.844105 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:24.844575 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:24.845006 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:24.845691 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:24.846136 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:24.846579 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:24.913455 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:24.921020 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-e2f26f0a-cb97-4007-9886-c9c963940ea0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1278 microversion: 2.1 time: 0.089287 Sep 18 01:11:24.921588 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 562/1128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:24 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1278 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:25.945298 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:25.946441 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:25.948942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:25.955417 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:25.955903 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:25.956342 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:25.957039 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:25.957494 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:25.957919 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:26.086740 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:26.094430 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-ab96f0af-5c0c-4e66-9fe6-ae9e1c7c59c6 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1318 microversion: 2.1 time: 0.153899 Sep 18 01:11:26.095022 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 567/1129] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:25 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:26.163296 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [req-0a08cada-6908-4a29-bc23-6cfde5a2e44a req-33ba5404-b66d-4ae7-aaca-ebc8a0666826 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f28a9b5c-488b-4b11-95e4-d701d660354f", "tag": "da6e6f7b-2f98-4561-a55a-fe284adeefaf"}]} {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:11:26.174536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0a08cada-6908-4a29-bc23-6cfde5a2e44a req-33ba5404-b66d-4ae7-aaca-ebc8a0666826 service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:26.175129 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0a08cada-6908-4a29-bc23-6cfde5a2e44a req-33ba5404-b66d-4ae7-aaca-ebc8a0666826 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:26.175732 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [req-0a08cada-6908-4a29-bc23-6cfde5a2e44a req-33ba5404-b66d-4ae7-aaca-ebc8a0666826 service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:26.188454 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.compute.server_external_events [req-0a08cada-6908-4a29-bc23-6cfde5a2e44a req-33ba5404-b66d-4ae7-aaca-ebc8a0666826 service nova] Creating event network-changed:da6e6f7b-2f98-4561-a55a-fe284adeefaf for instance f28a9b5c-488b-4b11-95e4-d701d660354f on hv-cinder-maste Sep 18 01:11:26.193383 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [req-0a08cada-6908-4a29-bc23-6cfde5a2e44a req-33ba5404-b66d-4ae7-aaca-ebc8a0666826 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034353 Sep 18 01:11:26.193979 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 563/1130] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:11:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:11:27.119663 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:27.120252 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:27.126699 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:27.127178 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:27.127608 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:27.128295 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:27.128742 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:27.129163 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:27.257172 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:27.264665 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-df3cd3c1-1551-43b0-8ed3-54efb6fc89ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1318 microversion: 2.1 time: 0.150353 Sep 18 01:11:27.265226 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 568/1131] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:27 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1318 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:28.287663 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:28.288275 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:28.294895 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:28.295368 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:28.295792 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:28.296466 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:28.296911 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:28.297329 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:28.429659 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:28.437407 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-efc62dc5-a38e-476e-80f3-cf59640175f0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1318 microversion: 2.1 time: 0.154079 Sep 18 01:11:28.437973 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 564/1132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:28 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:29.461224 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:29.461817 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:29.468221 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:29.468700 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:29.469167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:29.469851 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:29.470308 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:29.470758 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:29.602738 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:29.610306 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-caf3a9dd-c501-4525-adb5-1c1dd58b0e96 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1318 microversion: 2.1 time: 0.153650 Sep 18 01:11:29.610901 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 569/1133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:29 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:30.632697 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:30.633299 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:30.639731 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:30.640208 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:30.640639 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:30.641329 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:30.641774 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:30.642190 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:30.685906 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-b5adb48d-2e7c-4dd2-adc4-7432a197cbd0 req-1a4cc250-a90d-426b-b4de-4eb5ea27398b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f28a9b5c-488b-4b11-95e4-d701d660354f", "name": "network-vif-plugged", "status": "completed", "tag": "da6e6f7b-2f98-4561-a55a-fe284adeefaf"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:11:30.697075 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-b5adb48d-2e7c-4dd2-adc4-7432a197cbd0 req-1a4cc250-a90d-426b-b4de-4eb5ea27398b service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:30.698148 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-b5adb48d-2e7c-4dd2-adc4-7432a197cbd0 req-1a4cc250-a90d-426b-b4de-4eb5ea27398b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:30.699305 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-b5adb48d-2e7c-4dd2-adc4-7432a197cbd0 req-1a4cc250-a90d-426b-b4de-4eb5ea27398b service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:30.711783 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-b5adb48d-2e7c-4dd2-adc4-7432a197cbd0 req-1a4cc250-a90d-426b-b4de-4eb5ea27398b service nova] Creating event network-vif-plugged:da6e6f7b-2f98-4561-a55a-fe284adeefaf for instance f28a9b5c-488b-4b11-95e4-d701d660354f on hv-cinder-maste Sep 18 01:11:30.717059 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-b5adb48d-2e7c-4dd2-adc4-7432a197cbd0 req-1a4cc250-a90d-426b-b4de-4eb5ea27398b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034834 Sep 18 01:11:30.717983 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 570/1134] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:11:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:11:30.791113 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:30.798626 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-884f062d-d83f-47cc-b7d0-f3cd4104bb0e tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1318 microversion: 2.1 time: 0.169826 Sep 18 01:11:30.799186 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 565/1135] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:30 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1318 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:31.822187 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:31.822942 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:31.829103 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:31.829596 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:31.830027 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:31.830741 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:31.831198 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:31.831622 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:31.961582 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:31.969154 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-7581eb5a-079a-43e4-bb8e-fb7fec96abc3 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1318 microversion: 2.1 time: 0.151721 Sep 18 01:11:31.969711 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 571/1136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:31 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:32.994143 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:32.994777 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:33.001172 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:33.001653 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:33.002092 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:33.002802 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:33.003250 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:33.003667 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:33.129684 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:33.137697 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-5024e173-f587-45db-9a0d-6a8d4dab7c6f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1507 microversion: 2.1 time: 0.147833 Sep 18 01:11:33.138259 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 566/1137] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:32 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1507 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:33.160167 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8aa43f5a-36ae-442b-b9ea-7a047d39e006", "device": "/dev/sdb"}} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:11:33.163035 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:33.169363 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:33.169845 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:33.170279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:33.170997 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:33.171452 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:33.171878 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:33.425008 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-41933b9d-5303-4c0c-8c93-59b96463dc32 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "POST /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.268588 Sep 18 01:11:33.425800 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 572/1138] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 18 01:11:33 2022] POST /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f/os-volume_attachments => generated 194 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:11:50.718696 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:50.719208 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:50.721297 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=2 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:50.721940 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell0:SELECT=6 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:50.722712 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:50.723704 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG dbcounter [-] [120832] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120832) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:50.731361 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:50.731954 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:50.732384 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:50.733068 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:50.733514 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:50.733934 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:50.919391 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.objects.instance [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lazy-loading 'services' on Instance uuid f28a9b5c-488b-4b11-95e4-d701d660354f {{(pid=120832) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 18 01:11:50.991835 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-bad86e40-6a4f-4701-9e79-382c9d1271ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "DELETE /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f/os-volume_attachments/8aa43f5a-36ae-442b-b9ea-7a047d39e006" status: 202 len: 0 microversion: 2.1 time: 0.276975 Sep 18 01:11:50.992442 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 567/1139] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 18 01:11:50 2022] DELETE /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f/os-volume_attachments/8aa43f5a-36ae-442b-b9ea-7a047d39e006 => generated 0 bytes in 278 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 18 01:11:52.113688 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell0:SELECT=2 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:52.114484 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_api:SELECT=24 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:52.115407 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG dbcounter [-] [120831] Writing DB stats nova_cell1:SELECT=37,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120831) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 18 01:11:52.120193 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:52.120782 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:52.128674 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:52.129158 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:52.129596 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:52.130279 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:52.130761 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:52.131694 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:52.157624 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Going to try to terminate instance {{(pid=120831) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 18 01:11:52.229203 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-02afc6d9-bb9f-421d-840b-36d542bb4d1f tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "DELETE /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 204 len: 0 microversion: 2.1 time: 0.116730 Sep 18 01:11:52.229805 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 573/1140] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 18 01:11:52 2022] DELETE /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 18 01:11:52.241257 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:52.241946 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:52.247997 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:52.248473 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:52.248902 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:52.249584 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:52.250032 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:52.250448 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:52.375546 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:52.383744 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-c302cb54-4358-40fa-b16b-000cea3a27ab tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1513 microversion: 2.1 time: 0.149245 Sep 18 01:11:52.384305 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 568/1141] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:52 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1513 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:53.406719 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:53.407411 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.compute.api [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120831) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:53.413766 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:53.414240 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:53.414709 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:53.415387 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:53.415836 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:53.416260 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:53.545127 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.policy [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120831) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:53.555217 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [None req-25746a01-fecc-4c76-8acb-f46256ca03c0 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1513 microversion: 2.1 time: 0.152720 Sep 18 01:11:53.555785 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 574/1142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:53 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1513 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 18 01:11:54.577762 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:54.578395 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:54.585207 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:54.585682 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:54.586108 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:54.586936 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:54.587387 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:54.587804 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:54.719790 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.policy [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cbfab0fe9b4f4840a3ad9953d90e86bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4f90e9a8c164448a214481704ff18a2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120832) authorize /opt/stack/nova/nova/policy.py:199}} Sep 18 01:11:54.729409 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-729630ce-174c-45b6-9227-46b5d9cc73dd tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 200 len: 1513 microversion: 2.1 time: 0.155509 Sep 18 01:11:54.734497 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 569/1143] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:54 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 1513 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 18 01:11:54.740045 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG nova.api.openstack.wsgi [req-f1cb6f57-5067-453e-949a-e2219e069241 req-730e8e49-18ff-4ebf-a4ed-9d28f70fad4e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f28a9b5c-488b-4b11-95e4-d701d660354f", "name": "network-vif-deleted", "tag": "da6e6f7b-2f98-4561-a55a-fe284adeefaf"}]} {{(pid=120831) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 18 01:11:54.750051 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-f1cb6f57-5067-453e-949a-e2219e069241 req-730e8e49-18ff-4ebf-a4ed-9d28f70fad4e service nova] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:54.750531 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-f1cb6f57-5067-453e-949a-e2219e069241 req-730e8e49-18ff-4ebf-a4ed-9d28f70fad4e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:54.750961 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: DEBUG oslo_concurrency.lockutils [req-f1cb6f57-5067-453e-949a-e2219e069241 req-730e8e49-18ff-4ebf-a4ed-9d28f70fad4e service nova] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120831) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:54.761735 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.compute.server_external_events [req-f1cb6f57-5067-453e-949a-e2219e069241 req-730e8e49-18ff-4ebf-a4ed-9d28f70fad4e service nova] Creating event network-vif-deleted:da6e6f7b-2f98-4561-a55a-fe284adeefaf for instance f28a9b5c-488b-4b11-95e4-d701d660354f on hv-cinder-maste Sep 18 01:11:54.766855 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: INFO nova.api.openstack.requestlog [req-f1cb6f57-5067-453e-949a-e2219e069241 req-730e8e49-18ff-4ebf-a4ed-9d28f70fad4e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030385 Sep 18 01:11:54.767757 dv-cinder-master-3-iscsi devstack@n-api.service[120831]: [pid: 120831|app: 0|req: 575/1144] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 18 01:11:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 18 01:11:55.753088 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Calling method '>' {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 18 01:11:55.753693 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.compute.api [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] [instance: f28a9b5c-488b-4b11-95e4-d701d660354f] Fetching instance by UUID {{(pid=120832) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 18 01:11:55.760147 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:55.760622 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:55.761053 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:55.761763 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Acquiring lock "e1feb29a-fb47-481b-b99a-721bee8616c1" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 18 01:11:55.762210 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 18 01:11:55.762662 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG oslo_concurrency.lockutils [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Lock "e1feb29a-fb47-481b-b99a-721bee8616c1" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120832) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 18 01:11:55.775536 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.wsgi [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] HTTP exception thrown: Instance f28a9b5c-488b-4b11-95e4-d701d660354f could not be found. Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Request method failure captured: Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: request: GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f HTTP/1.1 Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept: application/json Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Accept-Encoding: identity Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Connection: close Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Length: 0 Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Content-Type: application/json Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Host: 10.222.0.42 Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: Openstack-System-Scope: None Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: User-Agent: python-urllib3/1.26.12 Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Auth-Token: gAAAAABjJm_8DovFyWm-yip1DbAfFHpStHj6B0b-ZKXPx4ZHj3eBHWyq5Lh56Nb035XnRXCDtYZmn4qlI-tRWf9E8V9pSqAWdzNyXahHmfa9EEwkgEqs86nUnPfyJduYIcJkGL3CYh2NqYvxPudEmqKzjbrx5XGJ23ZXaUkl4sD2wr_aTHmcRhE Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Id: None Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Domain-Name: None Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Identity-Status: Confirmed Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Is-Admin-Project: True Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Id: default Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Domain-Name: Default Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Id: a4f90e9a8c164448a214481704ff18a2 Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1303083818 Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Role: reader,member Sep 18 01:11:55.777141 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Roles: reader,member Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/a4f90e9a8c164448a214481704ff18a2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/a4f90e9a8c164448a214481704ff18a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.42:8080", "publicURL": "http://10.222.0.42:8080/v1/AUTH_a4f90e9a8c164448a214481704ff18a2"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/a4f90e9a8c164448a214481704ff18a2"}]}] Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant: tempest-VolumesSnapshotTestJSON-1303083818 Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Id: a4f90e9a8c164448a214481704ff18a2 Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1303083818 Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User: tempest-VolumesSnapshotTestJSON-1303083818-project Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Id: default Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Domain-Name: Default Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Id: cbfab0fe9b4f4840a3ad9953d90e86bb Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: X-User-Name: tempest-VolumesSnapshotTestJSON-1303083818-project Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: method: > Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: exception: Instance f28a9b5c-488b-4b11-95e4-d701d660354f could not be found. Sep 18 01:11:55.778984 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: action_args: {'id': 'f28a9b5c-488b-4b11-95e4-d701d660354f'} Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: {{(pid=120832) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f28a9b5c-488b-4b11-95e4-d701d660354f could not be found. Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 18 01:11:55.780724 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f28a9b5c-488b-4b11-95e4-d701d660354f could not be found. Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f28a9b5c-488b-4b11-95e4-d701d660354f could not be found. Sep 18 01:11:55.782390 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: ERROR nova.api.openstack.wsgi Sep 18 01:11:55.783921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: DEBUG nova.api.openstack.wsgi [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] Returning 404 to user: Instance f28a9b5c-488b-4b11-95e4-d701d660354f could not be found. {{(pid=120832) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 18 01:11:55.783921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: INFO nova.api.openstack.requestlog [None req-7ecb2d46-1341-4970-9638-a681c5192198 tempest-VolumesSnapshotTestJSON-1303083818 tempest-VolumesSnapshotTestJSON-1303083818-project] 10.222.0.42 "GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f" status: 404 len: 111 microversion: 2.1 time: 0.029327 Sep 18 01:11:55.783921 dv-cinder-master-3-iscsi devstack@n-api.service[120832]: [pid: 120832|app: 0|req: 570/1145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 18 01:11:55 2022] GET /compute/v2.1/servers/f28a9b5c-488b-4b11-95e4-d701d660354f => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)